Installation¶
For the Mozilla setup, please have a look at the MozDef Mana page.
The installation process has been tested on CentOS 6 and RHEL 6.
Docker¶
You can quickly install MozDef with an automated build generation using docker.
Dockerfile¶
After installing docker, use this to build a new image:
cd docker && sudo make build
Running the container:
sudo make run
You’re done! Now go to:
- http://127.0.0.1:3000 < meteor (main web interface)
- http://127.0.0.1:9090 < kibana
- http://127.0.0.1:9200 < elasticsearch
- http://127.0.0.1:9200/_plugin/marvel < marvel (monitoring for elasticsearch)
- http://127.0.0.1:8080 < loginput
- http://127.0.0.1:8081 < rest api
Known issues¶
- Marvel doesn’t display node info: ` Oops! FacetPhaseExecutionException[Facet [fs.total.available_in_bytes]: failed to find mapping for fs.total.available_in_bytes]`
Marvel uses techniques to gather system info that are not compatible with docker. See https://groups.google.com/forum/#!topic/elasticsearch/dhpxaOuoZWI
Despite this issue, marvel runs fine.
- I don’t see any data or dashboards in Kibana
We need to create some sample data, it’s in our roadmap ;)
Elasticsearch nodes¶
This section explains the manual installation process for Elasticsearch nodes (search and storage).
ElasticSearch¶
Installation instructions are available on Elasticsearch website. You should prefer packages over archives if one is available for your distribution.
Marvel plugin¶
Marvel is a monitoring plugin developed by Elasticsearch (the company).
WARNING: this plugin is NOT open source. At the time of writing, Marvel is free for development but you have to get a license for production.
To install Marvel, on each of your elasticsearch node, from the Elasticsearch home directory:
bin/plugin -i elasticsearch/marvel/latest
sudo service elasticsearch restart
You should now be able to access to Marvel at http://any-server-in-cluster:9200/_plugin/marvel
Web and Workers nodes¶
This section explains the manual installation process for Web and Workers nodes.
Python¶
We need to install a python2.7 virtualenv:
sudo yum install make zlib-devel bzip2-devel openssl-devel ncurses-devel sqlite-devel readline-devel tk-devel pcre-devel gcc gcc-c++
cd
wget http://python.org/ftp/python/2.7.6/Python-2.7.6.tgz
tar xvzf http://python.org/ftp/python/2.7.6/Python-2.7.6.tgz
./configure --prefix=/home/mozdef/python2.7 --enable-shared
make
make install
wget https://raw.github.com/pypa/pip/master/contrib/get-pip.py
export LD_LIBRARY_PATH=/home/netantho/python2.7/lib/
./python2.7/bin/python get-pip.py
./python2.7/bin/pip install virtualenv
mkdir ~/envs
cd ~/envs
~/python2.7/bin/virtualenv mozdef
source mozdef/bin/activate
pip install -r MozDef/requirements.txt
At this point when you launch python, It should tell you that you’re using Python 2.7.6.
Whenever you launch a python script from now on, you should have your mozdef virtualenv actived and your LD_LIBRARY_PATH env variable should include /home/mozdef/python2.7/lib/
RabbitMQ¶
RabbitMQ is used on workers to have queues of events waiting to be inserted into the Elasticsearch cluster (storage).
To install it, first make sure you enabled EPEL repos. Then you need to install an Erlang environment:
yum install erlang
You can then install the rabbitmq server:
rpm --import http://www.rabbitmq.com/rabbitmq-signing-key-public.asc
yum install rabbitmq-server-3.2.4-1.noarch.rpm
To start rabbitmq at startup:
chkconfig rabbitmq-server on
Meteor¶
Meteor is a javascript framework used for the realtime aspect of the web interface.
We first need to install Mongodb since it’s the DB used by Meteor. In /etc/yum.repo.d/mongo, add:
[mongodb]
name=MongoDB Repository
baseurl=http://downloads-distro.mongodb.org/repo/redhat/os/x86_64/
gpgcheck=0
enabled=1
Then you can install mongodb:
sudo yum install mongodb
For meteor, in a terminal:
curl https://install.meteor.com/ | sh
wget http://nodejs.org/dist/v0.10.26/node-v0.10.26.tar.gz
tar xvzf node-v0.10.26.tar.gz
cd node-v0.10.26
./configure
make
make install
Make sure you have meteorite/mrt:
npm install -g meteorite
Then from the meteor subdirectory of this git repository run:
mrt add iron-router
mrt add accounts-persona
You may want to edit the app/lib/settings.js file to properly point to your elastic search server:
elasticsearch={
address:"http://servername:9200/",
healthurl:"_cluster/health",
docstatsurl:"_stats/docs"
}
Then start meteor with:
meteor
Nginx¶
We use nginx webserver.
You need to install nginx:
sudo yum install nginx
If you don’t have this package in your repos, before installing create /etc/yum.repos.d/nginx.repo with the following content:
[nginx]
name=nginx repo
baseurl=http://nginx.org/packages/centos/6/$basearch/
gpgcheck=0
enabled=1
UWSGI¶
We use uwsgi to interface python and nginx:
wget http://projects.unbit.it/downloads/uwsgi-2.0.2.tar.gz
~/python2.7/bin/python uwsgiconfig.py --build
~/python2.7/bin/python uwsgiconfig.py --plugin plugins/python core
cp python_plugin.so ~/envs/mozdef/bin/
cp uwsgi ~/envs/mozdef/bin/
cd rest
# modify uwsgi.ini
vim uwsgi.ini
uwsgi --ini uwsgi.ini
cd ../loginput
# modify uwsgi.ini
vim uwsgi.ini
uwsgi --ini uwsgi.ini
sudo cp nginx.conf /etc/nginx
# modify /etc/nginx/nginx.conf
sudo vim /etc/nginx/nginx.conf
sudo service nginx restart
Kibana¶
Kibana is a webapp to visualize and search your Elasticsearch cluster data:
wget https://download.elasticsearch.org/kibana/kibana/kibana-3.0.0milestone5.tar.gz
tar xvzf kibana-3.0.0milestone5.tar.gz
mv kibana-3.0.0milestone5 kibana
# configure /etc/nginx/nginx.conf to target this folder
sudo service nginx reload
Import dashboards from MozDef/kibana/dashboards into the kibana webUI