The child's father was reportedly supposed to drop the little boy off at nursery in the morning, but apparently forgot the child was in the car.
"He was constantly saying 'What have I done, what have I done?'"
He then drove to work and locked the child inside the vehicle, failing to check the backseat as he did so.
It wasn't until the man left work and started driving home that he looked in the backseat and saw the child was still strapped in the car seat and unresponsive.
Dale Hamilton, who works with local police, told American news site CBS46 that he saw the man drive into a car park, jump out and begin CPR on his baby.
He said: "He hopped out of his car. I didn't know what he was doing at the time.
"But, he eventually pulled the child out of the car seat. I guess he was trying to un-restrain him. He pulled him out of the car seat, laid him on the ground and was trying to resuscitate him.
"He was constantly saying 'What have I done, what have I done.'"
The police officer added: "It is tough, I am not sure how someone could forget they have a child in the backseat."
When police arrived on the scene, the father ad to be restrained due to his distress.
No charges have been filed, but he could face charges of involuntary manslaughter.
The heat in Atlanta on Wednesday reached 90 degrees. Police estimate that the heat inside the car could have easily reached 130 to 140 degrees during the day.