Step 2. Configure such per the supplier.
As you can imagine I won't be able to pick or support such but my bet is you really didn't want a proxy server as it's not like a connection to a router or your ISP at all. However you did ask for a PROXY SERVER so I must take your word you know what you are asking for.
https://www.wingate.com/ seems to be free to use for up to 10 users today.
My ISP allows me to get multiple public IPs from a single modem. There is only one ethernet port on the modem, but if I connect 5 devices to it via a switch, each will have it's own public ipv4 address. IPs are dynamic but stay reserved for a while to the last accessing MAC address.
I want to dedicate a pc to act as a proxy server, allowing remote connection to these IPs.
Please give me your suggestions how can I do this, as my networking knowledge is not that good and I have no idea how to approach this. I'd prefer to make this on Windows OS without coding, but any information and guidance will be of great help!