With very little investigation, I discovered a lengthy article about configuring Pelican, Dropbox, and automatic blog publishing using Dropbox. Well I'm not using Dropbox, I've a different setup, but it would still be handy to watch a folder and run a script when changes were found. This is something I epect I'll come back to again and again, so it makes sense to make my own notes.
I'm most interested in [using watcher to regenerate the blog automatically] and in the words of Roberto Zoia:
Watcher is a nifty utility written in Python that monitors a directory and its subdirectories for change using the inotify interface. It can be configured to execute a command when a change is detected, which is just what we need to regenerate our blog.
Sounds perfect, doesn't it. Rather than copying his article for my reference, I'll follow it and make notes here so I'll be able to reproduce the configuration at a later date.
Terminal commands, can be run in a virtualenv or not. The first thing, is that Andreas Gohr created a forked version, noted in Roberto Zoia's article which uses an .ini file instead of a YAML file. I'm not sure what the benefit of that is, but let's go with what worked for Roberto.
$ pip install pyinotify
$ wget https://github.com/splitbrain/Watcher/archive/master.zip
$ unzip master.zip
$ mv ./Watcher-master/* .
$ rmdir ./Watcher-master
$ chmod +x ./watcher.py
The configuration options in this particular version of watcher, are in an ini file called watcher.ini, stored alongside the watcher.py script.
Um, well, that didn't work did it. Haha, this happens to me a lot, let's investigate the issue.
$ python /home/<user>/<project>/watcher.py -c /home/nllewellyn/lilnick.co.uk/watcher.ini start
File "/home/<user>/<project>/watcher.py", line 68
except OSError, e:
SyntaxError: invalid syntax
It may be that, according to the documentation, there is no option to have watcher.ini in it's own folder.
See the provided watcher.ini file for an example job configuration. The config file should reside in /etc/watcher.ini or ~/.watcher.ini. You can also specify the path to the config file as a command line parameter using the --config option.
If you edit the ini file you must restart the daemon for it to reload the configuration.
Let's move it into /etc/watcher.ini and see what happens then, before we go rummaging around in the code to find out what is going wrong.
$ sudo mv ./watcher.ini /etc/watcher.ini
$ python /home/nllewellyn/lilnick.co.uk/watcher.py start File "/home/<user>/<project>/watcher.py", line 68
except OSError, e:
SyntaxError: invalid syntax
Well, that didn't work either. Let's start looking at the code of watcher.py and see what is going on in line 68.
57 def daemonize(self):
59 do the UNIX double-fork magic, see Stevens' "Advanced Programming in the
60 UNIX Environment" for details (ISBN 0201563177)
64 pid = os.fork()
65 if pid > 0:
66 #exit first parent
68 except OSError, e:
69 sys.stderr.write("fork #1 failed: %d (%s)\n" % (e.errno, e.strerror))
72 # decouple from parent environment
Well it looks like OSError is a standard procedure, and Python is complaining about something being written incorrectly in the syntax, or usage of this. A little searching reveals, that in Python 2 both, the comma and as were permitted on this function. It makes sense that this is a fault then, I'm using Python 3.6.7 and Andreas Gohr said this was written for Python 2.7.
There are two ways of fixing this, the first way would be to alter the code and the second way would be to alter the script to use the older version of Python. I currently have Python 2.7.15+ installed alongside Python 3.6.7. The simplest thing here would be to move the watcher.ini file back, and force the use of Python 2.7.
$ sudo mv /etc/watcher.ini ./watcher.ini
$ python2 /home/<user>/<project>/watcher.py -c /home/<user>/<project>/watcher.ini start
Traceback (most recent call last):
File "/home/nllewellyn/lilnick.co.uk/watcher.py", line 36, in <module>
ImportError: No module named pyinotify
Well now then I've not got pyinotify installed into Python 2.7, let's do that as it is a dependency, and try again.
$ pip2 install pyinotify
Cache entry deserialization failed, entry ignored
Installing collected packages: pyinotify
Successfully installed pyinotify-0.9.6
$ python2 /home/nllewellyn/lilnick.co.uk/watcher.py -c /home/nllewellyn/lilnick.co.uk/watcher.ini start
$ python2 /home/nllewellyn/lilnick.co.uk/watcher.py -c /home/nllewellyn/lilnick.co.uk/watcher.ini status
Sheesh, that was long-winded and incomplete, after a little testing I noticed a silly error. I asked watcher to watch the output folder, when it should have been watching the content folder doh! Let's alter that watcher.in file, and restart that watcher service.
$ python2 /home/nllewellyn/lilnick.co.uk/watcher.py -c /home/nllewellyn/lilnick.co.uk/watcher.ini stop
Edit that watcher.ini file and alter the line watch=/home/nllewellyn/lilnick.co.uk/output to watch=/home/nllewellyn/lilnick.co.uk/content instead and try again.
Incredible, it works. I save a file into output and the Pelican publish scripts run. Wonderful. Now all I need to do is run through this process again to tidy it all up for my other sites, and host the whole lot within a simple container of some sort on a server always online where I can update the content files remotely suing something like Syncthing to move the new content to the central location.
Fantastic. I'm chuffed to bits about this.