You're only supposed to log time when you are "manipulating the controls" i.e., PIC/SIC or as a rated crewmember. Safety pilots and many others can log time without touching the controls, but as a passenger, you have no role in the operation of the aircraft, and therefore cannot log time.

Under FAA regulations, you cannot log flight time toward a certificate or rating if you are flying strictly as a passenger. "Logging" time is a legal record of aeronautical experience, and it is reserved for pilots who are the "sole manipulator of the controls" or are serving as "required crew members" (such as a safety pilot or flight instructor). If you are a pilot flying in the passenger seat of a friend's plane, you can certainly record the flight in a personal journal for sentimental reasons, but it must not be added to your official pilot logbook totals. Misrepresenting passenger time as flight time can be considered a fraudulent entry, which can lead to the suspension or revocation of a pilot's license if discovered during a checkride or FAA inspection.