I'd use a HOSTS file. To learn about this, and I'm guessing the machines use Windows here are the words for google.
WINDOWS HOSTS FILE
I have few computers connected to my router + an iphone
I am running an http server on one of the computers
I can access my pages on that computer using the local ip, but I am looking for a way to do it using a name instead
I tried to assign that computer a name and a static ip on the router, but that did not do anything. I also put a dns server on that computer, then change the dns number on my router, but that did not work also.