I just bought my first ever laptop. It is a HP machine.
It comes with a 65W AC adapter. The first time I used the it(for about an hour) , the adapter got fairly hot. Is this normal ? Or should I consider returning it.
Laptop AC Adapter bricks get fairly warm. They can certainly feel hot when touched. In general, anything taking AC and converting to DC will generate a significant amount of heat, especially under load. Laptop power supplies are usually a Linear Power Supply, fairly inefficient but also inexpensive. They waste a lot of energy in the form of heat.
Consider: 120V @1.5 A (180W) AC is converted into 20V @3.25A (65W). The input power is far less than output, and some of that was lost to heat.
The 65W 20V adapter for my Lenovo T61 is always warm/hot at the end of shift, regardless of the battery charge status.
Don't keep them buried under pillows or clothes or paper, give them space to transfer that heat energy to the air.