Phone cameras are great for snapping a selfie, or posting a picture of that colorful margarita you ordered for Happy Hour. But, they do have limitations. They're terrible in low light. Almost none of them have optical zoom, so they're not good for some long range situations. Having to hover your finger over the screen can be awkward. No manual controls in dynamic situations. And, in cases where you care about the quality, a phone camera is usually best described as mediocre. Etc.
Personally, I use my phone when taking a quick snapshot that I plan to send to someone. But, I also carry a medium-level pocket camera, a Canon S100. Mostly because I used to do amateur photography, and I like having a camera with an optical zoom and easy access to manual functions. Also, my pocket camera can be out of my pocket and ready to take pictures in far less time than unlocking my phone and swiping to the camera app.
I know people who say that they feel like you do. "Why would I want to carry a camera?" And then they complain to me about how the picture they took has a bunch of noisy colored dots. Or, their subject was so far away that it's only a few pixels tall in the picture. And those pixels are fuzzy, because the massive number of pixels of the sensor are wasted because of the physical resolving ability of a lens the size of a phone camera.
But when I try to explain it, they wave it off. "I don't care about that stuff."
Phone cameras have their place. So do dedicated cameras. Use the one that fits your budget and your needs.