At least 20 years i guess
Edit: The earliest English essay on recreational fishing (also known as sport fishing) was published in 1496.
Big-game fishing started as a sport after the invention of the motorized boat. In 1898
So i think it is considered as a sport for a long time. There are sports that don't really need that much activity as football, basketball or so. Chess is a sport too for example.