Monday, July 7, 2014

SELinux problems with Erlang on Fedora/RHEL7

couchdb is one of several widely used server applications written in erlang.  I was surprised to find hundreds of Bugzilla tickets for erlang services and selinux-policy.

Problem #1: Erlang is treated by SELinux as part of the RabbitMQ Service
Erlang is a general purpose VM like Java.  However selinux-policy treated core components of Erlang as part of the rabbitmq semodule.

https://github.com/TresysTechnology/refpolicy-contrib/blob/master/rabbitmq.fc
/usr/lib/erlang/erts.*/bin/beam.*       --      gen_context(system_u:object_r:rabbitmq_beam_exec_t,s0)
/usr/lib/erlang/erts.*/bin/epmd --      gen_context(system_u:object_r:rabbitmq_epmd_exec_t,s0)

Apparently someone attempted to make SELinux work only for rabbitmq, erroneously thinking that the generic erlang runtime binaries are part of rabbitmq.

# ls -lZ /usr/lib64/erlang/erts-5.10.4/bin 
-rwxr-xr-x. root root system_u:object_r:rabbitmq_beam_exec_t:s0 beam
-rwxr-xr-x. root root system_u:object_r:rabbitmq_beam_exec_t:s0 beam.smp
-rwxr-xr-x. root root system_u:object_r:lib_t:s0       child_setup
-rwxr-xr-x. root root system_u:object_r:lib_t:s0       ct_run
-rwxr-xr-x. root root system_u:object_r:lib_t:s0       dialyzer
-rwxr-xr-x. root root system_u:object_r:lib_t:s0       dyn_erl
-rwxr-xr-x. root root system_u:object_r:rabbitmq_epmd_exec_t:s0 epmd
-rwxr-xr-x. root root system_u:object_r:lib_t:s0       erl
-rwxr-xr-x. root root system_u:object_r:lib_t:s0       erlc
-rwxr-xr-x. root root system_u:object_r:lib_t:s0       erlexec
-rwxr-xr-x. root root system_u:object_r:lib_t:s0       erl.src
-rwxr-xr-x. root root system_u:object_r:lib_t:s0       escript
-rwxr-xr-x. root root system_u:object_r:lib_t:s0       heart
-rwxr-xr-x. root root system_u:object_r:lib_t:s0       inet_gethost
-rwxr-xr-x. root root system_u:object_r:lib_t:s0       run_erl
-rwxr-xr-x. root root system_u:object_r:lib_t:s0       start
-rw-r--r--. root root system_u:object_r:lib_t:s0       start_erl.src
-rw-r--r--. root root system_u:object_r:lib_t:s0       start.src
-rwxr-xr-x. root root system_u:object_r:lib_t:s0       to_erl
-rwxr-xr-x. root root system_u:object_r:lib_t:s0       typer

Everything in this directory should be context bin_t.

Problem #2: Erlang Services Are Not Using Their Own Semodules!
The many erlang-related tickets filed against selinux-policy are confused because they are not actually running with the expected SELinux context.

# systemctl start  rabbitmq-server.service
# ps auxZ |grep rabbitmq
system_u:system_r:init_t:s0     rabbitmq   359 11.5  0.4 2177032 38972 ?       Ssl  20:18   0:01 /usr/lib64/erlang/erts-5.10.4/bin/beam.smp -W w -K true -A30 -P 1048576 -- -root /usr/lib64/erlang -progname erl -- -home /var/lib/rabbitmq -- -pa /usr/lib/rabbitmq/lib/rabbitmq_server-3.1.5/sbin/../ebin -noshell -noinput -s rabbit boot -sname rabbit@server -boot start_sasl -kernel inet_default_connect_options [{nodelay,true}] -sasl errlog_type error -sasl sasl_error_logger false -rabbit error_logger {file,"/var/log/rabbitmq/rabbit@server.log"} -rabbit sasl_error_logger {file,"/var/log/rabbitmq/rabbit@server-sasl.log"} -rabbit enabled_plugins_file "/etc/rabbitmq/enabled_plugins" -rabbit plugins_dir "/usr/lib/rabbitmq/lib/rabbitmq_server-3.1.5/sbin/../plugins" -rabbit plugins_expand_dir "/var/lib/rabbitmq/mnesia/rabbit@server-plugins-expand" -os_mon start_cpu_sup false -os_mon start_disksup false -os_mon start_memsup false -mnesia dir "/var/lib/rabbitmq/mnesia/rabbit@server"

The service is running as init_t. This is because  /usr/lib/rabbitmq/bin/rabbitmq-server is not properly labeled with something like rabbitmq_exec_t so it never transitions into its own semodule when launched from systemd.

Similarly couchdb.service launched itself via ExecStart=/usr/bin/erl (symlink /usr/lib64/erlang/erts-5.10.4/bin) and it too was running as init_t.  Lacking execve to anything labeled couchdb_exec_t, erl would fail to transition into  couchdb_t.  couchdb-1.6.0-9+ now runs from systemd with ExecStart=/usr/libexec/couchdb <parameters>.  If the wrapper shell script /usr/libexec/couchdb is labeled as couchdb_exec_t then couchdb service actually runs as couchdb_t.

Note: couchdb-1.6.0-9+ does not use the upstream /bin/couchdb script from systemd because it is rather broken and upstream plans on removing it in the next release.  The way we configure and exeucte couchdb from systemd is close to the future upstream standard way of handling the service.  While we do not use it for the systemd service it remains in the $PATH because users may depend on its behavior for debugging.

Temporary Workaround for CouchDB + SELinux
couchdb-1.6.0-9 may be the first Erlang application in Fedora to properly be confined by its own semodule(?)  We are waiting for selinux-policy to be fixed.  Meanwhile this temporary workaround will allow CouchDB to operate with SELinux enforcing enabled.  These instructions have been tested on Fedora 20 and RHEL 7.

Here are tested packages that I use on RHEL7 + EPEL7.  EPEL7 will soon have these packages.

#### Upgrade to couchdb-1.6.0-9 or later.
# sudo yum install -y selinux-policy-devel git
# git clone https://github.com/wtogami/refpolicy-contrib ||:
# cd refpolicy-contrib
# git checkout couchdb
# git pull
# make -f /usr/share/selinux/devel/Makefile couchdb.pp
# semodule -i couchdb.pp
# sudo restorecon -Rv /usr/lib64/erlang/erts-*/bin/
# sudo restorecon -v /usr/libexec/couchdb 
# sudo restorecon -v /usr/lib64/erlang/lib/couch-*/priv/couchjs
# sudo restorecon -v /usr/lib/systemd/system/couchdb.service

TODO's
Erlang Service Packages
In general erlang packages with their own semodule must execute via a properly labeled wrapper script.  /usr/bin/erl on its own is unable to guess and transition into the proper SELinux context.  See couchdb-1.6.0-9+ /usr/libexec/couchdb for an example.  As noted above, as rabbitmq never was running with the proper context someone who is familiar with rabbitmq will need to ensure the policy continues to operate as expected.

SELinux Policy
https://bugzilla.redhat.com/show_bug.cgi?id=1116014
Generally, selinux-policy must no longer treat core Erlang binaries as part of rabbitmq.  Everything in the /usr/lib*/erlang/erts-*/bin/ directory should be bin_t, perhaps somewhere in the core policies.

Every server application with its own semodule will need the aforementioned wrapper script(s) with context <something>_exec_t.  For example these rabbitmq binaries need to be properly labeled.

ls -Z /usr/lib/rabbitmq/lib/rabbitmq_server-3.1.5/sbin
-rwxr-xr-x. root root system_u:object_r:lib_t:s0       rabbitmqctl
-rwxr-xr-x. root root system_u:object_r:lib_t:s0       rabbitmq-defaults
-rwxr-xr-x. root root system_u:object_r:lib_t:s0       rabbitmq-env
-rwxr-xr-x. root root system_u:object_r:lib_t:s0       rabbitmq-plugins
-rwxr-xr-x. root root system_u:object_r:lib_t:s0       rabbitmq-server

I fixed the couchdb semodule to work with couchdb-1.6.0-8+.  rabbitmq and other packages may need specific fixes to their policies and perhaps also to their packages.

Changelog
July 9th, 2014
  • Updated couchdb policy and improved workaround instructions.
  • couchdb-1.6.0-9.fc20 coming soon for F19 and F20, and EL7.
  • Updated couchdb policy below
    • Fixed all AVC's in diskup
    • Added missing restorecon to workaround instructions.
    • Added unix_dgram_socket create needed for systemd notify