Large towels are not permitted in the hot springs, but you can bring a small towel in with you. You can use the small towel to wash your body before heading into the water or wipe your sweat while soaking in the onsen. And if you are feeling shy, you can even use it to cover up, though it's admittedly not much.