Usage¶
Web Interface¶
MozDef uses the Meteor framework for the web interface and bottle.py for the REST API. For authentication, MozDef ships with native support for Persona. Meteor (the underlying UI framework) also supports many authentication options including google, github, twitter, facebook, oath, native accounts, etc.
Events visualizations¶
Since the backend of MozDef is Elastic Search, you get all the goodness of Kibana with little configuration. The MozDef UI is focused on incident handling and adding security-specific visualizations of SIEM data to help you weed through the noise.
Alerts¶
Alerts are implemented as Elastic Search searches. MozDef provides a plugin interface to allow open access to event data for enrichment, hooks into other systems, etc.
Incident handling¶
Sending logs to MozDef¶
Events/Logs are accepted as json over http(s) with the POST or PUT methods or over rabbit-mq. Most modern log shippers support json output. MozDef is tested with support for:
- heka
- beaver
- nxlog
- logstash
- native python code
- AWS cloudtrail (via native python)
We have some configuration snippets
What should I log?¶
If your program doesn’t log anything it doesn’t exist. If it logs everything that happens it becomes like the proverbial boy who cried wolf. There is a fine line between logging too little and too much but here is some guidance on key events that should be logged and in what detail.
Event | Example | Rationale |
---|---|---|
Authentication Events | Failed/Success logins | Authentication is always an important event to log as it establishes traceability for later events and allows correlation of user actions across systems. |
Authorization Events | Failed attempts to insert/update/delete a record or access a section of an application. | Once a user is authenticated they usually obtain certain permissions. Logging when a user’s permissions do not allow them to perform a function helps troubleshooting and can also be helpful when investigating security events. |
Account Lifecycle | Account creation/deletion/update | Adding, removing or changing accounts are often the first steps an attacker performs when entering a system. |
Password/Key Events | Password changed, expired, reset. Key expired, changed, reset. | If your application takes on the responsibility of storing a user’s password (instead of using a centralized source) it is important to note changes to a users credentials or crypto keys. |
Account Activations | Account lock, unlock, disable, enable | If your application locks out users after failed login attempts or allows for accounts to be inactivated, logging these events can assist in troubleshooting access issues. |
Application Exceptions | Invalid input, fatal errors, known bad things | If your application catches errors like invalid input attempts on web forms, failures of key components, etc creating a log record when these events occur can help in troubleshooting and tracking security patterns across applications. Full stack traces should be avoided however as the signal to noise ratio is often overwhelming. It is also preferable to send a single event rather than a multitude of events if it is possible for your application to correlate a significant exception. For example, some systems are notorious for sending a connection event with source IP, then sending an authentication event with a session ID then later sending an event for invalid input that doesn’t include source IP or session ID or username. Correctly correlating these events across time is much more difficult than just logging all pieces of information if it is available. |
JSON format¶
This section describes the structure JSON objects to be sent to MozDef. Using this standard ensures developers, admins, etc are configuring their application or system to be easily integrated into MozDef.
Background¶
Mozilla used CEF as a logging standard for compatibility with Arcsight and for standardization across systems. While CEF is an admirable standard, MozDef prefers JSON logging for the following reasons:
- Every development language can create a JSON structure
- JSON is easily parsed by computers/programs which are the primary consumer of logs
- CEF is primarily used by Arcsight and rarely seen outside that platform and doesn’t offer the extensibility of JSON
- A wide variety of log shippers (heka, logstash, fluentd, nxlog, beaver) are readily available to meet almost any need to transport logs as JSON.
- JSON is already the standard for cloud platforms like amazon’s cloudtrail logging
Description¶
As there is no common RFC-style standard for json logs, we prefer the following structure adapted from a combination of the graylog GELF and logstash specifications.
Note all fields are lowercase to avoid one program sending sourceIP, another sending sourceIp, another sending SourceIPAddress, etc. Since the backend for MozDef is elasticsearch and fields are case-sensitive this will allow for easy compatibility and reduce potential confusion for those attempting to use the data. MozDef will perform some translation of fields to a common schema but this is intended to allow the use of heka, nxlog, beaver and retain compatible logs.
Mandatory Fields¶
Field | Purpose | Sample Value |
---|---|---|
category | General category/type of event matching the ‘what should I log’ section below | Authentication, Authorization, Account Creation, Shutdown, Startup, Account Deletion, Account Unlock, brointel, bronotice |
details | Additional, event-specific fields that you would like included with the event. Please completely spell out a field rather an abbreviate: i.e. sourceipaddress instead of srcip. | “dn”: “john@example.com,o=com, dc=example”, “facility”: “daemon” |
hostname | The fully qualified domain name of the host sending the message | server1.example.com |
processid | The PID of the process sending the log | 1234 |
processname | The name of the process sending the log | myprogram.py |
severity | RFC5424 severity level of the event in all caps: DEBUG, INFO, NOTICE, WARNING, ERROR, CRITICAL, ALERT, EMERGENCY | INFO |
source | Source of the event (file name, system name, component name) | /var/log/syslog/2014.01.02.log |
summary | Short human-readable version of the event suitable for IRC, SMS, etc. | john login attempts over threshold, account locked |
tags | An array or list of any tags you would like applied to the event | vpn, audit nsm,bro,intel |
timestamp | Full date plus time timestamp of the event in ISO format including the timezone offset | 2014-01-30T19:24:43+00:00 |
Details substructure (mandatory if such data is sent, otherwise optional)¶
Field | Purpose | Sample Value |
---|---|---|
destinationipaddress | Destination IP of a network flow | 8.8.8.8 |
destinationport | Destination port of a network flow | 80 |
sourceipaddress | Source IP of a network flow | 8.8.8.8 |
sourceport | Source port of a network flow | 42297 |
sourceuri | Source URI such as a referer | https://www.mozilla.org/ |
destinationuri | Destination URI as in “wget this URI” | https://www.mozilla.org/ |
error | Action resulted in an error or failure | true/false |
username | Username, email, login, etc. | kang@mozilla.com |
useragent | Program agent string | curl/1.76 (Windows; 5.1) |
Examples¶
{
"timestamp": "2014-02-14T11:48:19.035762739-05:00",
"hostname": "somemachine.in.your.company.com",
"processname": "/path/to/your/program.exe",
"processid": 3380,
"severity": "INFO",
"summary": "joe login failed",
"category": "authentication",
"source": "ldap",
"tags": [
"ldap",
"adminAccess",
"failure"
],
"details": {
"username": "joe",
"task": "access to admin page /admin_secret_radioactiv",
"result": "10 authentication failures in a row"
}
}
Writing alerts¶
Alerts allow you to create notifications based on events stored in elasticsearch. You would usually try to aggregate and correlate events that are the most severe and on which you have response capability. Alerts are stored in the alerts folder.
There are two types of alerts:
- simple alerts that consider events on at a time. For example you may want to get an alert everytime a single LDAP modification is detected.
- aggregation alerts allow you to aggregate events on the field of your choice. For example you may want to alert when more than 3 login attempts failed for the same username.
To narrow the events your alert sees, you need to specify filters. You can either use pyes to do that or load them from a Kibana dashboard.
You’ll find documented examples in the alerts folder.
Once you’ve written your alert, you need to configure it in celery to be launched periodically.
If you have a AlertBruteforceSsh
class in a alerts/bruteforce_ssh.py
file for example, in alerts/lib/config
you can configure the task to run every minute:
ALERTS = {
'bruteforce_ssh.AlertBruteforceSsh': crontab(minute='*/1'),
}