InstallingMediaGoblin

I'm using servus.local, Fedora 18 64 bit running on a laptop in the ""server room"".

Starting from http://docs.mediagoblin.org/siteadmin/deploying.html
 * and /usr/share/doc/postgresql-9.2.3/README.rpm-dist

= install yum packages for everything except transcoding = yum install python-paste-deploy python-paste-script git-core python python-devel python-lxml python-imaging python-virtualenv -y yum install postgresql-server postgresql python-psycopg2 -y yum install numpy -y
 * 1) For celeryd audio media type

= install of gcc, required for sqlalchemy speedups (next stage, setup.py develop) = yum install gcc -y

= enable rpmfusion for ffmpeg and install gstreamer plugins for transcoding stuff = (the error below doesn't happen if you have these installed) su -c 'yum localinstall --nogpgcheck http://download1.rpmfusion.org/free/fedora/rpmfusion-free-release-$(rpm -E %fedora).noarch.rpm http://download1.rpmfusion.org/nonfree/fedora/rpmfusion-nonfree-release-$(rpm -E %fedora).noarch.rpm' yum install gstreamer-python gstreamer-plugins-good gstreamer-plugins-base gstreamer-plugins-bad gstreamer-plugins-ugly gstreamer-ffmpeg -y

= setup the postgresql database = postgresql-setup initdb systemctl enable posgresql.service systemctl start postgresql.service

= create mediagoblin user, nologin = useradd mediagoblin vim /etc/passwd # change /bin/bash to /bin/nologin

= finished setup of postgresql =

= verify psql works with it = su mediagoblin /bin/bash psql (you should get a prompt)

= rest of instructions until further notice done as mediagoblin user = (i.e. don't quit the previous su, whoami should say mediagoblin)

= clone and setup virtualenv = git clone git://gitorious.org/mediagoblin/mediagoblin.git cd mediagoblin (virtualenv --system-site-packages . || virtualenv .) && ./bin/python setup.py develop

= creating our own config. enable plugins, turn on emails, use correct postgresql hostname. = cp mediagoblin.ini mediagoblin_local.ini vim mediagoblin_local.ini

= initialize database = ./bin/gmg dbupdate

= install nginx = yum install -y nginx yum install -y selinux-policy-devel setsebool httpd_can_network_connect on semanage permissive -a httpd_t
 * 1) required for "semanage permissive" (otherwise you get a failure for missing Makefile)
 * 1) enable port forwarding, socket connections from httpd context are disabled by default
 * 1) This part is a hack - it's slightly better then making everything permissive (taken from man selinux_http)

You can get nginx to access static files by any permission fix, since there was a mediagoblin group and nginx user I added the nginx user to the mediagoblin group and so the default permissions and ownership that mediagoblin sets on the transcoded files worked (660 for mediagoblin:mediagoblin): vim /etc/groups find mediagoblin append nginx
 * 1) or equivalently: useradd -G mediagoblin nginx

= create systemd unit file to start celeryd /lib/systemd/system/gmg-celeryd.service = [Unit] Description=Mediagoblin Celeryd [Service] User=mediagoblin Group=mediagoblin Type=simple WorkingDirectory=/home/mediagoblin/mediagoblin Environment="MEDIAGOBLIN_CONFIG=/home/mediagoblin/mediagoblin/mediagoblin_local.ini" \ CELERY_CONFIG_MODULE=mediagoblin.init.celery.from_celery ExecStart=/home/mediagoblin/mediagoblin/bin/celeryd PIDFile=/var/run/mediagoblin/mediagoblin-celeryd.pid [Install] WantedBy=multi-user.target
 * 1)             --daemon
 * 2)             --server-name=fcgi fcgi_host=127.0.0.1 fcgi_port=26543' mediagoblin

= create logging directory = mkdir /var/run/mediagoblin chown mediagoblin:mediagoblin /var/run/mediagoblin

= create systemd unit file to start paster: /lib/systemd/system/gmg.service = [Unit] Description=Mediagoblin

[Service] Type=forking User=mediagoblin Group=mediagoblin Environment=CELERY_ALWAYS_EAGER=false WorkingDirectory=/home/mediagoblin/mediagoblin ExecStart=/home/mediagoblin/mediagoblin/bin/paster serve \ /home/mediagoblin/mediagoblin/paste.ini \ --pid-file=/var/run/mediagoblin/mediagoblin.pid \ --log-file=/var/run/mediagoblin/mediagoblin.log \ --daemon \ --server-name=fcgi fcgi_host=127.0.0.1 fcgi_port=26543 ExecStop=/home/mediagoblin/mediagoblin/bin/paster serve \ --pid-file=/var/run/mediagoblin/mediagoblin.pid \ /home/mediagoblin/mediagoblin/paste.ini stop PIDFile=/var/run/mediagoblin/mediagoblin.pid [Install] WantedBy=multi-user.target

= enable and run them, check them = systemctl enable gmg-celeryd systemctl enable gmg systemctl start gmg-celeryd systemctl start gmg

= configure nginx to proxy to paster: use instructions in mediagoblin guide. =

= tmpfiles: on fedora /var is now a tmpfs = so you need to create the run directory on start. That is done either by the app or by tmpfiles.d. d /var/run/mediagoblin 755 mediagoblin mediagoblin
 * 1) cat /etc/tmpfiles.d/gmg.conf

= running the transcoding as mediagoblin:mediagoblin = the files are produced with rx- --- --- permissions, so nginx running as nginx:nginx cannot read them. There is probably a better solution (using setfacl maybe, I couldn't get it to work), but right now a hack on mediagoblin itself (I could also run a background process to change permissions..) to change permissions does the trick.

= open http and https (TODO: document http to https redirection and certificate setup with nginx): = firewall-cmd --add-service http firewall-cmd --add-service https

= reboot and verify everything still works. =

= Postgres instructions =

create user mediagoblin;

\dg # show all roles, including mediagoblin, has no roles, and member of nothing

create database mediagoblin with owner = mediagoblin;

postgres=# \dg List of roles Role name |                   Attributes                   | Member of -++--- mediagoblin |                                               | {}

postgres=# \l List of databases Name    |    Owner    | Encoding |   Collate   |    Ctype    |   Access privileges -+-+--+-+-+--- mediagoblin | mediagoblin | UTF8    | en_US.UTF-8 | en_US.UTF-8 |

After "./bin/gmg dbupdate" you should have the following tables:

mediagoblin=> \dt

List of relations Schema |          Name            | Type  |    Owner +---+---+- public | core__attachment_files   | table | mediagoblin public | core__collection_items   | table | mediagoblin public | core__collections        | table | mediagoblin public | core__file_keynames      | table | mediagoblin public | core__media_comments     | table | mediagoblin public | core__media_entries      | table | mediagoblin public | core__media_tags         | table | mediagoblin public | core__mediafiles         | table | mediagoblin public | core__migrations         | table | mediagoblin public | core__processing_metadata | table | mediagoblin public | core__tags               | table | mediagoblin public | core__users              | table | mediagoblin public | image__mediadata         | table | mediagoblin public | video__mediadata         | table | mediagoblin (14 rows)

= Bugs = Transcording problem, this was before adding rpmfusion and installing gstreamer-ffmpeg, gstreamer-plugins-bad, gstreamer-plugins-ugly:

2013-03-16 15:55:33,412 DEBUG  [mediagoblin.processing.task] Processing  2013-03-16 15:55:33,415 INFO   [mediagoblin.media_types.video.transcoders] Initializing VideoTranscoder... 2013-03-16 15:55:33,416 INFO   [mediagoblin.media_types.video.transcoders] Discovering /home/mediagoblin/mediagoblin/user_dev/media/queue/media_entries/bce350d1-9d89-4ee3-b1c0-15193b90fb88/13030052.mp4 Traceback (most recent call last): File "/home/mediagoblin/mediagoblin/mediagoblin/media_types/video/transcoders.py", line 726, in __on_discovered raise Exception('Could not discover {0}'.format(self.source_path)) Exception: Could not discover /home/mediagoblin/mediagoblin/user_dev/media/queue/media_entries/bce350d1-9d89-4ee3-b1c0-15193b90fb88/13030052.mp4 2013-03-16 15:55:33,445 INFO   [mediagoblin.media_types.video.transcoders] Terminating MainLoop 2013-03-16 15:55:33,446 ERROR  [mediagoblin.processing.task] An unhandled exception was raised while processing  2013-03-16 15:55:33,447 WARNING [mediagoblin.processing] No idea what happened here, but it failed: TypeError("argument of type 'NoneType' is not iterable",) 2013-03-16 15:55:33,502 WARNING [mediagoblin.processing] No idea what happened here, but it failed: TypeError("argument of type 'NoneType' is not iterable",) Error - : argument of type 'NoneType' is not iterable URL: http://servus.local:6543/submit/ File '/usr/lib/python2.7/site-packages/paste/exceptions/errormiddleware.py', line 144 in __call__ app_iter = self.application(environ, sr_checker) File '/usr/lib/python2.7/site-packages/paste/urlmap.py', line 203 in __call__ return app(environ, start_response) File '/home/mediagoblin/mediagoblin/lib/python2.7/site-packages/Beaker-1.5.4-py2.7.egg/beaker/middleware.py', line 152 in __call__ return self.wrap_app(environ, session_start_response) File '/home/mediagoblin/mediagoblin/mediagoblin/app.py', line 236 in __call__ return self.call_backend(environ, start_response) File '/home/mediagoblin/mediagoblin/mediagoblin/app.py', line 216 in call_backend response = controller(request) File '/home/mediagoblin/mediagoblin/mediagoblin/decorators.py', line 48 in new_controller_func return controller(request, *args, **kwargs) File '/home/mediagoblin/mediagoblin/mediagoblin/submit/views.py', line 96 in submit_start run_process_media(entry, feed_url) File '/home/mediagoblin/mediagoblin/mediagoblin/submit/lib.py', line 68 in run_process_media task_id=entry.queued_task_id) File '/home/mediagoblin/mediagoblin/lib/python2.7/site-packages/celery-2.5.3-py2.7.egg/celery/app/task/__init__.py', line 445 in apply_async  return self.apply(args, kwargs, task_id=task_id, **options) File '/home/mediagoblin/mediagoblin/lib/python2.7/site-packages/celery-2.5.3-py2.7.egg/celery/app/task/__init__.py', line 601 in apply   request=request, propagate=throw) File '/home/mediagoblin/mediagoblin/lib/python2.7/site-packages/celery-2.5.3-py2.7.egg/celery/execute/trace.py', line 248 in eager_trace_task uuid, args, kwargs, request) File '/home/mediagoblin/mediagoblin/lib/python2.7/site-packages/celery-2.5.3-py2.7.egg/celery/execute/trace.py', line 181 in trace_task  R = retval = fun(*args, **kwargs) File '/home/mediagoblin/mediagoblin/mediagoblin/processing/task.py', line 92 in run   entry.media_manager['processor'](proc_state) File '/home/mediagoblin/mediagoblin/mediagoblin/media_types/video/processing.py', line 91 in process_video   store_metadata(entry, metadata) File '/home/mediagoblin/mediagoblin/mediagoblin/media_types/video/processing.py', line 170 in store_metadata   if key in metadata]) TypeError: argument of type 'NoneType' is not iterable CGI Variables -  CONTENT_LENGTH: '7497175' CONTENT_TYPE: 'multipart/form-data; boundary=---185490103311094336141507406595' CSRF_TOKEN: u'18270768167913598379' HTTP_ACCEPT: 'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8' HTTP_ACCEPT_ENCODING: 'gzip, deflate' HTTP_ACCEPT_LANGUAGE: 'en-US,en;q=0.5' HTTP_CONNECTION: 'keep-alive' HTTP_COOKIE: 'mediagoblin_csrftoken=18270768167913598379; mediagoblin_csrftoken=18270768167913598379; mediagoblin=57de25a836140d810f8c9d5ed58d2bc9' HTTP_DNT: '1' HTTP_HOST: 'servus.local:6543' HTTP_REFERER: 'http://servus.local:6543/submit/' HTTP_USER_AGENT: 'Mozilla/5.0 (X11; Linux x86_64; rv:19.0) Gecko/20100101 Firefox/19.0' PATH_INFO: '/submit/' REMOTE_ADDR: '10.81.2.190' REQUEST_METHOD: 'POST' SERVER_NAME: '0.0.0.0' SERVER_PORT: '6543' SERVER_PROTOCOL: 'HTTP/1.1' WSGI Variables --  application: {(None, ''): , (None, '/mgoblin_static'): , (None, '/theme_static'): , (None, '/mgoblin_media'): } beaker.get_session: > beaker.session: {'user_id': u'1', '_accessed_time': 1363442133.226504, 'messages': [], '_creation_time': 1363439039.214583} paste.httpserver.thread_pool:  paste.throw_errors: True werkzeug.request:  wsgi process: 'Multithreaded'