It is not that easy for a newbie to make a workable VPS by himself.
I have three year experience on VPS. I can make it as simple as possible to show you how to do it in three steps.
0) THIS STEP IS NOT COUNTED.
You need a VPS at least. I recommend BurstNet. The price is good. The service is OK per price.
After you order your own VPS and paid the invoice, you probably get the email in 8 hours for your account information.
You have panel and ssh account.
Went to panel and rebuild your VPS by choosing Centos 5 minimum OS template.
Now it is my three steps
1) Step 1
SSH your VPS by root user. Enter following command to install httpd
yum install httpd-devel
2) Step 2
Install MySQL server, and start MySQL server.
yum install mysql mysql-server mysql-devel
3) Step 3
Install PHP and Restart Apache
yum install php php-mysql php-common php-gd php-mbstring php-mcrypt php-devel php-xml
Now you VPS has Linux ( Centos), Apache web server, MySQL database, PHP support. LAMP.
It is so simple, you have your VPS down in 5 minutes and three steps.
To manage your VPS, you may want to add a free web control panel. I use webmin for all my VPS servers.