Usage

Web Interface

MozDef uses the Meteor framework for the web interface and bottle.py for the REST API. For authentication, MozDef ships with native support for Persona. Meteor (the underlying UI framework) also supports many authentication options including google, github, twitter, facebook, oath, native accounts, etc.

Events visualizations

Since the backend of MozDef is Elastic Search, you get all the goodness of Kibana with little configuration. The MozDef UI is focused on incident handling and adding security-specific visualizations of SIEM data to help you weed through the noise.

Alerts

Alerts are generally implemented as Elastic Search searches, or realtime examination of the incoming message queues. MozDef provides a plugin interface to allow open access to event data for enrichment, hooks into other systems, etc.

Incident handling

Sending logs to MozDef

Events/Logs are accepted as json over http(s) with the POST or PUT methods or over rabbit-mq. Most modern log shippers support json output. MozDef is tested with support for:

We have some configuration snippets

What should I log?

If your program doesn’t log anything it doesn’t exist. If it logs everything that happens it becomes like the proverbial boy who cried wolf. There is a fine line between logging too little and too much but here is some guidance on key events that should be logged and in what detail.

Event Example Rationale
Authentication Events Failed/Success logins Authentication is always an important event to log as it establishes traceability for later events and allows correlation of user actions across systems.
Authorization Events Failed attempts to insert/update/delete a record or access a section of an application. Once a user is authenticated they usually obtain certain permissions. Logging when a user’s permissions do not allow them to perform a function helps troubleshooting and can also be helpful when investigating security events.
Account Lifecycle Account creation/deletion/update Adding, removing or changing accounts are often the first steps an attacker performs when entering a system.
Password/Key Events Password changed, expired, reset. Key expired, changed, reset. If your application takes on the responsibility of storing a user’s password (instead of using centralized LDAP/persona) it is important to note changes to a users credentials or crypto keys.
Account Activations Account lock, unlock, disable, enable If your application locks out users after failed login attempts or allows for accounts to be inactivated, logging these events can assist in troubleshooting access issues.
Application Exceptions Invalid input, fatal errors, known bad things

If your application catches errors like invalid input attempts on web forms, failures of key components, etc creating a log record when these events occur can help in troubleshooting and tracking security patterns across applications. Full stack traces should be avoided however as the signal to noise ratio is often overwhelming.

It is also preferable to send a single event rather than a multitude of events if it is possible for your application to correlate a significant exception.

For example, some systems are notorious for sending a connection event with source IP, then sending an authentication event with a session ID then later sending an event for invalid input that doesn’t include source IP or session ID or username. Correctly correlating these events across time is much more difficult than just logging all pieces of information if it is available.

JSON format

This section describes the structure JSON objects to be sent to MozDef. Using this standard ensures developers, admins, etc are configuring their application or system to be easily integrated into MozDef.

Background

Mozilla used CEF as a logging standard for compatibility with Arcsight and for standardization across systems. While CEF is an admirable standard, MozDef prefers JSON logging for the following reasons:

  • Every development language can create a JSON structure
  • JSON is easily parsed by computers/programs which are the primary consumer of logs
  • CEF is primarily used by Arcsight and rarely seen outside that platform and doesn’t offer the extensibility of JSON
  • A wide variety of log shippers (heka, logstash, fluentd, nxlog, beaver) are readily available to meet almost any need to transport logs as JSON.
  • JSON is already the standard for cloud platforms like amazon’s cloudtrail logging

Description

As there is no common RFC-style standard for json logs, we prefer the following structure adapted from a combination of the graylog GELF and logstash specifications.

Note all fields are lowercase to avoid one program sending sourceIP, another sending sourceIp, another sending SourceIPAddress, etc. Since the backend for MozDef is elasticsearch and fields are case-sensitive this will allow for easy compatibility and reduce potential confusion for those attempting to use the data. MozDef will perform some translation of fields to a common schema but this is intended to allow the use of heka, nxlog, beaver and retain compatible logs.

Mandatory Fields

Field Purpose Sample Value
category General category/type of event matching the ‘what should I log’ section below Authentication, Authorization, Account Creation, Shutdown, Startup, Account Deletion, Account Unlock, brointel, bronotice
details Additional, event-specific fields that you would like included with the event. Please completely spell out a field rather an abbreviate: i.e. sourceipaddress instead of srcip. “dn”: “john@example.com,o=com, dc=example”, “facility”: “daemon”
hostname The fully qualified domain name of the host sending the message server1.example.com
processid The PID of the process sending the log 1234
processname The name of the process sending the log myprogram.py
severity RFC5424 severity level of the event in all caps: DEBUG, INFO, NOTICE, WARNING, ERROR, CRITICAL, ALERT, EMERGENCY INFO
source Source of the event (file name, system name, component name) /var/log/syslog/2014.01.02.log
summary Short human-readable version of the event suitable for IRC, SMS, etc. john login attempts over threshold, account locked
tags An array or list of any tags you would like applied to the event

vpn, audit

nsm,bro,intel

timestamp Full date plus time timestamp of the event in ISO format including the timezone offset 2014-01-30T19:24:43+00:00

Details substructure (optional fields)

Field Purpose Used In Sample Value
destinationipaddress Destination IP of a network flow NSM/Bro/Intel 8.8.8.8
destinationport Destination port of a network flow NSM/Bro/Intel 80
dn Distinguished Name in LDAP, mean unique ID in the ldap hierarchy event/ldap john@example.org,o=org, dc=example
filedesc   NSM/Bro/Intel  
filemimetype   NSM/Bro/Intel  
fuid   NSM/Bro/Intel  
result Result of an event, success or failure event/ldap LDAP_SUCCESS
seenindicator Intel indicator that matched as seen by our system NSM/Bro/Intel evil.com/setup.exe
seenindicator_type Type of intel indicator NSM/Bro/Intel HTTP::IN_URL
seenwhere Where the intel indicator matched (which protocol, which field) NSM/Bro/Intel Intel::URL
source Source of the connection event/ldap

Mar 19 15:36:25 ldap1 slapd[31031]: conn=6633594 fd=49 ACCEPT from IP=10.54.70.109:23957 (IP=0.0.0.0:389)

Mar 19 15:36:25 ldap1 slapd[31031]: conn=6633594 op=0 BIND

sourceipaddress Source IP of a network flow

NSM/Bro/Intel

event/ldap

8.8.8.8
sourceport Source port of a network flow NSM/Bro/Intel 42297
sources Source feed NSM/Bro/Intel CIF - need-to-know
success Auth success event/ldap True
uid Bro connection uid NSM/Bro/Intel CZqhEs40odso1tFNx3

Examples

{
    "timestamp": "2014-02-14T11:48:19.035762739-05:00",
    "hostname": "fedbox",
    "processname": "/tmp/go-build278925522/command-line-arguments/_obj/exe/log_json",
    "processid": 3380,
    "severity": "INFO",
    "summary": "joe login failed",
    "category": "authentication",
    "source": "",
    "tags": [
        "MySystem",
        "Authentication"
    ],
    "details": {
        "user": "joe",
        "task": "access to admin page /admin_secret_radioactiv",
        "result": "10 authentication failures in a row"
    }
}