The former world heavyweight champion passed away at a care facility in Arizona on Wednesday at the age of 70, having suffered congestive heart failure.
The American great was involved in a string of memorable bouts, and was considered something of a bogeyman for the great Muhammad Ali.
The pair were involved in three epic fights, Norton winning the first on a split decision after breaking Ali's jaw, before Ali won the second, also on a split decision, and the third, when he was given a very controversial verdict.
Nelson described Norton as "tough, through and through, and very well-respected in the boxing world".
He added: "He was a bogeyman for a lot of fighters. He was very tough to dissect and get through so, unless you were an outstanding fighter, you struggled against him.
"He boxed the best, Larry Holmes, Earnie Shavers, Jimmy Young and, of course Ali and George Foreman. This guy was tough, an ex-marine.
"If you look on Twitter and the Internet, there are not many, if any, people who had a bad word to say about him.
"Ken Norton was the gatekeeper to the heavyweight division. If he boxed in today's era, he'd be a champion, without a doubt."
Norton utilised quite an unusual style and Nelson added: "It was sort of a cross-block style, very unconventional in regards to boxing so, if you had somebody like that in front of you, you thought 'this isn't what I've trained to try and get through'.
"Norton came in with that style, and he was very athletic with it as well, and very physical as well, so he could make it very hard for an opponent to figure him out and get the timing.
"Ali was one of the greatest fighters we've ever seen but he still struggled with this man and his speed, his jab, his style, his fitness.
"All of a sudden, Ali had to depend on his skill more than anything else.
"You kind of felt sorry for Norton because you thought 'you know what, you boxed Ali three times and they should have given you the mark on every occasion, because you made life hard for him'. It was close and they gave the nod to Ali."
Norton was also involved in a classic fight with Larry Holmes in 1978, losing his title on a split decision after an enthralling contest.
"One of the greatest heavyweight fights you could ever see," said Nelson. "15 rounds of an immense, class fight, and it was the passing of the baton. That fight had everything, it was second to none."