Single quote in the password to the database during Moodle installation breaks the process

I have setup a database named `moodle' which a user named `moodle' identified by password `Let's Learn!' will use.

In one of the dialog during the installation process that asked me for the database settings, I entered `Let's Learn!' as the password.

Later after there was a notification that `config.php' had been created successfully, the installation process stopped.

Checking the last line in the error log, there is a line saying:
[error] [client 127.0.0.1] PHP Parse error: syntax error, unexpected T_STRING in /var/www/htdocs/moodle/config.php on line 9

Looking at `config.php', I found the following line:
$CFG->dbpass = 'Let's Learn!';

After I changed the above line to:
$CFG->dbpass = 'Let\'s Learn!';
I can continue the installation process by opening `index.php' once again.