Tap water (“eau du robinet”) is always safe to drink, unless you have a sign that says otherwise (“eau non potable” is the most common), which is often the case in public bathrooms for instance. Note that in most restaurants for instance, tap water is always an option (“une carafe d'eau”).