Im not an IT person per say but I think its idiotic to type 500 characters into a command line in place of 5 mouse clicks
That's a major exaggeration. When I moved our site to this current server, all I was given was the barebones Debian install (which, at this point, uses about 80MB of RAM). From there, it likely took no more than 250 characters to have the site fully operational. Those ~250 characters encapsulate:
Updating the OS
Creating a user
Creating a secure SSH tunnel
Installing extra required tools
Syncing the website from the old server to the new one
Creating SQL users
Creating and importing the SQL databases
Editing the configuration for Apache and other tools
Fin
The only front-end on the server is for our SQL management (which is way, way easier than using the CLI for most things) - but that's Web-based, much like WordPress is. As a general rule, I like servers that are as light-weight as possible... doesn't have tools and software I won't be using. It means there is less chance of a security hole, as well. We don't even install an FTP server... because we don't need it, and it's one less potential issue down the road.
I find a GUI server OS easier to manage from a cell phone
That has nothing to do with the OS itself. I am 100% fine with services like that running on the server. It's the desktop portion that's ridiculous to me. It's not just overhead, it's the fact that there is a complete desktop environment for a server that should never have to be accessed physically. It's a SERVER! What I am trying to say is, even without a desktop environment, you could still connect to it through graphical tools (like we do for our SQL).
I truly believe that if you master a CLI, you will be more efficient in taking care of server configuration and trouble-shooting than with a GUI. I don't have experience with Windows Server, but I sure as heck have a lot of experience with tweaking and using Windows in general.
This of course is all thrown out the window if typing in general doesn't come naturally to you.