The situation
- You’ve installed
ghoston your server (in my case centos) - You’ve installed
forever - You’re running
ghostthroughforeveras root. - The only problem is you don’t want to run it as root as it is not a good idea to be running ghost, or any public app as root or with root priveleges. It is best to make another user who will run ghost. We will name such a user ghost.
So we leave forever running ghost in root and do a quick
$su ghost
$forever start index.js
Expecting everything to work as before but instead I get hit with one of these:
warn: --minUptime not set. Defaulting to: 1000ms
warn: --spinSleepTime not set. Your script will exit if it does not stay up for at least 1000ms
info: Forever processing file: /var/www/ghost/index.js
fs.js:427
return binding.open(pathModule._makeLong(path), stringToFlags(flags), mode);
^
Error: ENOENT, no such file or directory '/home/ghost/.forever/2hWg.log'
at Object.fs.openSync (fs.js:427:18)
at Object.forever.startDaemon (/usr/lib/node_modules/forever/lib/forever.js:406:14)
at /usr/lib/node_modules/forever/lib/forever/cli.js:257:13
at /usr/lib/node_modules/forever/lib/forever/cli.js:144:5
at /usr/lib/node_modules/forever/lib/forever.js:358:11
at Object.oncomplete (fs.js:107:15)
Argh! What’s this. Let’s try something else:
$forever list
/usr/lib/node_modules/forever/lib/forever.js:674
procs.forEach(function (proc) {
^
TypeError: Object Error: EACCES, mkdir '/home/ghost' has no method 'forEach'
at Object.forever.format (/usr/lib/node_modules/forever/lib/forever.js:674:11)
at /usr/lib/node_modules/forever/lib/forever.js:511:28
at /usr/lib/node_modules/forever/lib/forever.js:136:14
at /usr/lib/node_modules/forever/lib/forever.js:89:20
at /usr/lib/node_modules/forever/node_modules/utile/node_modules/mkdirp/index.js:25:29
at /usr/lib/node_modules/forever/node_modules/utile/node_modules/mkdirp/index.js:25:29
at /usr/lib/node_modules/forever/node_modules/utile/node_modules/mkdirp/index.js:37:53
at Object.oncomplete (fs.js:107:15)
Ok fine let’s try just node and see if maybe the problem is in forever
$node index.js
events.js:72
throw er; // Unhandled 'error' event
^
Error: listen EADDRINUSE
at errnoException (net.js:904:11)
at Server._listen2 (net.js:1042:14)
at listen (net.js:1064:10)
at net.js:1146:9
at dns.js:72:18
at process._tickDomainCallback (node.js:459:13)
at process._tickFromSpinner (node.js:390:15)
NOOO, even node isn’t working… but wait, the error looks a bit different now: EADDRINUSE. We forgot to turn off ghost as root. let’s go back and do that then.
$exit
$forever stopall
info: Forever stopped processes:
data: uid command script forever pid logfile uptime
data: [0] EFde /usr/bin/node /var/www/ghost/index.js 709 711 /root/.forever/EFde.log 0:4:51:46.864
$forever list
info: No forever processes running
Ok we are good to try again now.
$su ghost
$node index.js
Ghost is running in development...
Listening on YOUR.IP:2368
Url configured as: http://my-ghost-blog.com
Ctrl+C to shut down
Great. Let’s exit that and try forever again. We do the suggested Ctrl+C then run forever and … we get the same error as before. Argh! We are too flustered to give the error message more than a fleeting glance, do they actually tell us anything important? Ok fine, but just a quick glance, if we don’t look at the error hard it won’t be a hard error to fix… we secretly hope. The errors are centered around the directory /home/ghost. Let’s head over there and do some investigating into permissions.
$cd /home/ghost
bash: cd: /home/ghost: No such file or directory
Whoaaa it doesn’t even exist. Now that would definetely be a problem. What happened? I know ghost is a user, how come no home directory was made? Play dramatic music
The Reason
If you were following the ghost docs (great job), and you wanted to try the init option for starting ghost on startup then you would have executed the following commands:
$useradd -r ghost -U
$chown -R ghost:ghost /var/www/ghost
$chmod 755 /etc/init.d/ghost
Of course when I came to run the ubuntu script on centos I ran into many errors, but that is another post for another time. Anyways, the point is I have my ghost user now so there was no need to make a new one. The problem, however was in useradd -r ghost -U. I had taken a command from a different context and was hoping it would work in a the new context. Let’s break it down so we understand better.
From the man useradd command we find the following.
-r, --system
Create a system account.
System users will be created with no aging information in /etc/shadow, and their numeric identifiers are choosen in
the SYS_UID_MIN-SYS_UID_MAX range, defined in /etc/login.defs, instead of UID_MIN-UID_MAX (and their GID
counterparts for the creation of groups).
Note that useradd will not create a home directory for such an user, regardless of the default setting in
/etc/login.defs (CREATE_HOME). You have to specify the -m options if you want a home directory for a system account
to be created.
That last note is so critical. A home directory is not made because it is a system account. So what is a system account? Well I found this from gentoo wiki.
To enforce the separation of privileges, specific system accounts are created for each task. If you run a mail server on your system, that mail server will have a user account on your system.
These accounts are not usable by regular users: you can not log in on your system using those accounts. They exist only to allow the specific processes to run with their own permissions and privileges.
Ah so that’s what a system user is. Since another name for a home directory is the login directory (linfo.org), it makes sense that no home directory should be made for a system user as no one is meant to login using that account. So the ghost system account that was made earlier was supposed to run ghost, and it is indeed able to run ghost as we saw earlier. But what we tried to do was run forever and that’s where we hit the error. It turns out that forever uses the home directory to put its logs, and when it tries to do that it can’t because there is no such directory and ghost user does not have the privelege to make the directory (change anything in /home).
The Solution
Give ghost user a home directory
We can accomplish this in two ways.
Make the ghost user home directory manually
Let’s go back to root or a sudo enabled account and let’s execute the following commands.
$mkdir /home/ghost
$ls -l /home/
drwx------ 3 auser auser 4096 Dec 31 13:51 auser
drwxr-xr-x 2 root root 4096 Mar 31 12:13 ghost
drwx------ 2 buser buser 4096 Nov 11 23:31 buser
So we now have a ghost home directory but the persmissions and ownership are all wrong, as can be seen from the ls -l command. We remedy that by doing the following:
$chown ghost:ghost ghost
$chmod 700 /home/ghost
$chown ghost:ghost ghost
$ls -l
drwx------ 3 auser auser 4096 Dec 31 13:51 auser
drwx------ 2 ghost ghost 4096 Mar 31 12:13 ghost
drwx------ 2 buser buser 4096 Nov 11 23:31 buser
Now it looks proper again. Here’s a quick break down. With the chmod we made the permissions of the directory the same as the others. With the chown we made the user ghost and the group ghost owners of the directory ghost. Not confusing at all.
Delete ghost user and make him again the right way
Let’s just burn it all down and start again. First delete the user as root or a sudoer:
$userdel ghost
Note if you had already created the home root and want to try this way anyway add an r flag so it looks like this, userdel -r ghost
I’d be interested in seeing what happened to the ownership of the ghost directory. If you remember we had issued this command, $chown -R ghost:ghost /var/www/ghost earlier which gave the, now non-existant, ghost user ownership of the ghost directory and its contents. Let’s take a look at it now:
$ls -l /var/www/
drwxr-xr-x 6 498 498 4096 Apr 3 10:13 ghost
drwxr-xr-x 12 root root 4096 Nov 13 02:20 html
Where the name ghost used to be there are now the numbers 498. That was the UID (user ID) of ghost.
Now we add him again with the extra m flag.
$useradd -rm ghost -U
Check that the home directory was made:
$ls /home | grep ghost
ghost
Yup we have it. Now let’s check what happened to the ownership of the ghost directory:
$ls -l /var/www/
drwxr-xr-x 6 ghost ghost 4096 Apr 3 10:13 ghost
drwxr-xr-x 12 root root 4096 Nov 13 02:20 html
Looks like we were in luck and the new ghost had the same UID as the old one, so we don’t need to fix that.
Change the forever root (Not recommended because it is buggy as of version0.10.11)
First we need to figure out where the forever root is configured. Looking at the documentation of forever, I can’t seem to find what I am looking for. Time to take a look at the code. It just so happens that as I am looking at the github repo I notice that srossross had committed a fix where he added a FOREVER_ROOT variable. Great that’s just what we need. Only problem is it is so recent that the update is not available for update yet. As of version 0.10.11 it hadn’t been added yet. You can check the version you have using the following:
$npm list | grep forever@
forever@0.10.11 /usr/lib/node_modules/forever
So we will have to do this manually and change it ourselves. It is only one line so shouldn’t be too scary.
$vi /usr/lib/node_modules/forever/lib/forever.js
Go to the line that says something like,
forever.root = path.join(process.env.HOME || process.env.USERPROFILE || '/root', '.forever');
And just change it to the following:
forever.root = process.env.FOREVER_ROOT || path.join(process.env.HOME || process.env.USERPROFILE || '/root', '.forever');
Press ESC or in some cases Ctrl+c then :x to save in vi. Now let’s run the command with the new root as follows:
NODE_ENV=production FOREVER_ROOT=/var/www/ghost/forever forever start var/www/ghostindex.js
Now forever won’t need to have the home directory. As for the bug, you can check my issue I reported in github.
Testing the Solution
Now that I’ve solved the problem above, let’s test it out. Switch to the ghost user and run ghost as follows:
$su ghost
$forever start /var/www/ghost/index.js
warn: --minUptime not set. Defaulting to: 1000ms
warn: --spinSleepTime not set. Your script will exit if it does not stay up for at least 1000ms
info: Forever processing file: /var/www/ghost/index.js
Let’s check the log file just in case. We can see the log file by listing the forever processes as follows:
$ forever list
info: No forever processes running
NOOOO, how come? I just saw it saying Forever processing file: /var/www/ghost/index.js. Wait a sec, I think I’m in the wrong place.
$pwd
/root
Argh that’s why. I was in that directory as root and when switched to ghost it kept me there. Let’s go to somewhere more comfortable.
$cd
$pwd
/home/ghost
Alright try running it one more time and this time we do get a list.
info: Forever processes running
data: uid command script forever pid logfile uptime
data: [0] 7z38 /usr/bin/node /var/www/ghost/index.j 791 793 /home/ghost/.forever/7z38.log 0:0:0:5.952
Do a quick cat on that logfile and make sure its looking good
$cat /home/ghost/.forever/7z38.log
Ghost is running in development...
Listening on 192.241.121.253:2368
Url configured as: http://my-ghost-blog.com
Ctrl+C to shut down
That looks nice. Ok now let’s take a look at our blog and NOOOO all my old posts are gone. What happened? Did we lose all our content? Actually we ran it in development instead of production. Stop the process and run it again in development as follows:
$forever stopall
info: Forever stopped processes:
data: uid command script forever pid logfile uptime
data: [0] 7z38 /usr/bin/node /var/www/ghost/index.js 791 793 /root/.forever/7z38.log 0:0:0:9.299
$NODE_ENV=production forever start /var/www/ghost/index.js
warn: --minUptime not set. Defaulting to: 1000ms
warn: --spinSleepTime not set. Your script will exit if it does not stay up for at least 1000ms
info: Forever processing file: /var/www/ghost/index.js
Ok good that looks proper. Time to check the log file again. List and cat as before and we get:
ERROR: SQLITE_READONLY: attempt to write a readonly database, sql: update "settings" set "created_at" = ?, "created_by" = ?, "id" = ?, "key" = ?, "type" = ?, "updated_at" = ?, "updated_by" = ?, "uuid" = ?, "value" = ? where "id" = ?, bindings: Fri Mar 28 2014 21:55:58 GMT-0400 (EDT),1,16,installedApps,app,Thu Apr 03 2014 07:16:44 GMT-0400 (EDT),1,792297d0-2813-4e91-850e-578f38c76328,[],16
Error: SQLITE_READONLY: attempt to write a readonly database, sql: update "settings" set "created_at" = ?, "created_by" = ?, "id" = ?, "key" = ?, "type" = ?, "updated_at" = ?, "updated_by" = ?, "uuid" = ?, "value" = ? where "id" = ?, bindings: Fri Mar 28 2014 21:55:58 GMT-0400 (EDT),1,16,installedApps,app,Thu Apr 03 2014 07:16:44 GMT-0400 (EDT),1,792297d0-2813-4e91-850e-578f38c76328,[],16
at /var/www/ghost/node_modules/knex/clients/server/base.js:73:22
at tryCatch1 (/var/www/ghost/node_modules/knex/node_modules/bluebird/js/main/util.js:64:19)
at Promise$_callHandler [as _callHandler] (/var/www/ghost/node_modules/knex/node_modules/bluebird/js/main/promise.js:708:13)
at Promise$_settlePromiseFromHandler [as _settlePromiseFromHandler] (/var/www/ghost/node_modules/knex/node_modules/bluebird/js/main/promise.js:724:18)
at Promise$_settlePromiseAt [as _settlePromiseAt] (/var/www/ghost/node_modules/knex/node_modules/bluebird/js/main/promise.js:896:14)
at Promise$_rejectPromises [as _rejectPromises] (/var/www/ghost/node_modules/knex/node_modules/bluebird/js/main/promise.js:1033:14)
at Async$_consumeFunctionBuffer [as _consumeFunctionBuffer] (/var/www/ghost/node_modules/knex/node_modules/bluebird/js/main/async.js:64:12)
at Async$consumeFunctionBuffer (/var/www/ghost/node_modules/knex/node_modules/bluebird/js/main/async.js:37:14)
at Promise$_Scheduler (/var/www/ghost/node_modules/knex/node_modules/bluebird/js/main/schedule.js:40:58)
at process._tickDomainCallback (node.js:459:13)
ERROR: SQLITE_READONLY: attempt to write a readonly database, sql: update "settings" set "created_at" = ?, "created_by" = ?, "id" = ?, "key" = ?, "type" = ?, "updated_at" = ?, "updated_by" = ?, "uuid" = ?, "value" = ? where "id" = ?, bindings: Fri Mar 28 2014 21:55:58 GMT-0400 (EDT),1,16,installedApps,app,Thu Apr 03 2014 07:16:44 GMT-0400 (EDT),1,792297d0-2813-4e91-850e-578f38c76328,[],16
The app will not be loaded
Check with the app creator, or read the app documentation for more details on app requirements
Ghost is running...
Your blog is now available on http://blog.amayem.com
Ctrl+C to shut down
Possibly unhandled Error: SQLITE_READONLY: attempt to write a readonly database, sql: delete from "sessions" where "id" = ?, bindings: HwxKR0LJRcWG39FcT1ZwkMyq
at /var/www/ghost/node_modules/knex/clients/server/base.js:73:22
at tryCatch1 (/var/www/ghost/node_modules/knex/node_modules/bluebird/js/main/util.js:64:19)
at Promise$_callHandler [as _callHandler] (/var/www/ghost/node_modules/knex/node_modules/bluebird/js/main/promise.js:708:13)
at Promise$_settlePromiseFromHandler [as _settlePromiseFromHandler] (/var/www/ghost/node_modules/knex/node_modules/bluebird/js/main/promise.js:724:18)
at Promise$_settlePromiseAt [as _settlePromiseAt] (/var/www/ghost/node_modules/knex/node_modules/bluebird/js/main/promise.js:896:14)
at Promise$_rejectPromises [as _rejectPromises] (/var/www/ghost/node_modules/knex/node_modules/bluebird/js/main/promise.js:1033:14)
at Async$_consumeFunctionBuffer [as _consumeFunctionBuffer] (/var/www/ghost/node_modules/knex/node_modules/bluebird/js/main/async.js:64:12)
at Async$consumeFunctionBuffer (/var/www/ghost/node_modules/knex/node_modules/bluebird/js/main/async.js:37:14)
at Promise$_Scheduler (/var/www/ghost/node_modules/knex/node_modules/bluebird/js/main/schedule.js:40:58)
at process._tickDomainCallback (node.js:459:13)
NOOOO, more errors. But it was fine in development. Notice that in the middle of the errors we got a cool Ghost is running…, but of course we can’t continue with such an error. Let’s, grudingly, take a look at the error. Looks like a database issue, something is trying to write in a databse that is readonly. It could be forever or ghost, but judging by the path of the node modules where the error originated, /var/www/ghost/node_modules/knex/... it looks like ghost. There is an easy way to figure this out. Let’s run ghost without forever as follows (stop the forever process first though):
$NODE_ENV=production node /var/www/ghost/index.js
Aaaand we get the same error. Ok good so now we know it’s a ghost issue this time. Let’s go to this database and see its permissions:
$cd /var/www/ghost/content/data/
$ls -l
total 80
-rw-r--r-- 1 root root 35840 Apr 1 15:15 ghost.db
-rw-r--r-- 1 ghost ghost 31744 Apr 3 07:43 ghost-dev.db
-rw-r--r-- 1 ghost ghost 121 Mar 26 18:39 README.md
Aha! We found it. ghost.db is owned by root. The reason for that was that we had originally started ghost in production in root so the database was made with that ownership. The dev database was made in ghost because that was the first time I had run it in dev so there were no problems. The solution is simply to change the ownership from root or as a sudoer with sudo:
$chown ghost:ghost ghost.db
$ls -l
total 80
-rw-r--r-- 1 ghost ghost 35840 Apr 1 15:15 ghost.db
-rw-r--r-- 1 ghost ghost 31744 Apr 3 07:47 ghost-dev.db
-rw-r--r-- 1 ghost ghost 121 Mar 26 18:39 README.md
That looks much better. Let’s test again:
NODE_ENV=production node /var/www/ghost/index.js
Ghost is running...
Your blog is now available on http://blog.amayem.com
Ctrl+C to shut down
No more error and my posts are back. Solved!
