To access the "TFS web access", team expolorer installation is needed?

Yes and No

If the developer or tester has team explorer installed in thier systems, then they can use the TFS web Access URL for managing all other than the check-in check=-out code.

(or)

If the organization has the TFS CAL, then every one under CAL access the "TFS web access" with out installing the team explorer in individual systems. No need to get procured per seat 500$ for getting the web access.