No, the North Atlantic treaty does not cover any territory in Africa so Ceuta and Melilla are out of scope of NATO protection. The Canary Islands and Madeira are in the Atlantic north of the Tropic of Cancer so they are covered by NATO treaty protection.