Ninjas are infantry units that are very effective against Heroes (explorers, for example) and mercenaries. Ninjas themselves are mercenaries, so are easily countered by spies or other ninjas. The Ninja can also use Stealth Mode, which can be detected by an Explorer or Town Center (or other units).
The Ninja differs from the Spy in that it is classified as a mercenary, in contrast to a Spy, making the Spy very effective against Ninjas. The Ninja also has slightly less speed, higher cost, much lower Line of Sight and much lower multipliers, except against Native Warriors, but it has more hit points and much more (siege) attack. Overall, Ninjas are more well-rounded than Spies, and more deadly against Native Warriors and mercenaries, with the same effectiveness against heroes, and a lesser proficiency in scouting.
Ninjas are classified as Heavy infantry, and possess a high amount of melee resistance. While they are specialized at their anti-mercenary role, they are more than able to take on most melee units (with Jaguar Prowl Knights and Urumi being notable exceptions). They will beat all cavalry in the Commerce Age and most Veteran (or equivalent) cavalry, as well as melee infantry, such as Rodeleros and Doppelsoldners, due largely to their very high hit points and armour. They are very similar to Wokou Ronin in this regard. Their heavy infantry classification does put them at a disadvantage against skirmisher-type units, but gives them a strong advantage against units such as Sowars, which rely on multipliers to be effective; It is worth noting that Spies are countered heavily by Sowars and similar units.
The Ninja also has different accessibility compared to Spies, as Spies are always available from the Church/Mosque from the Colonial Age onward (except for the Dutch Church from the JapaneseConsulate), can be sent from the Home City by European civilizations, or 4 can be sent from the British once at the expense of, while the Ninja can sometimes be trained at the Saloon/Monastery, or sent with Home City shipments by certain civilizations.
Ninjas are very useful in the third and fourth scenarios of the Chinese campaign (by choosing Japanese Mercenaries at the Consulate) because most enemies on these maps (namely, Jaguar Warriors and Eagle Warriors) are classified as Native Warriors, which the Ninja has bonuses against.
The Ninjutsu-Zukai is identical to the Ninja, but are easier to access since they can be trained from the Japanese Consulate if Japanese Isolation is chosen. They start with +10% hit points and attack than the regular Ninja, and are automatically upgraded in every Age starting in the Fortress Age (see here for the exact values).
Shrouded by the mists of time, the exploits of the ninja continue to fire imaginations across the globe. During the turbulent years of Japan's sixth and seventh centuries, a secret class of warrior priests emerged. Rulers of opposing factions came to value the ninja for their skills in stealth, combat, and espionage.
As the centuries passed, the legendary powers of the ninja grew: invisibility, shape-shifting, flight; even the ability to pass through solid walls. Modern filmmakers blend historical fact with ancient myth to fashion a portrait of a warrior class that continues to fascinate and terrify.