A MAC address, also called physical address, is a unique identifier assigned to every network interfaces for communications on the physical network segment. In other words, you can identify the manufacturer of your device through your pyshical address.
There are different tools on the Internet that allow you to identify the manufacturer from the MAC Address. How in my 3 previous post I wrote about how to capture the wireless traffic and all MAC Address, now in this post I will explain how to implement a Docker container exposing a Rest API to get the Manufacturer from the captured MAC Address.
As everything should be lightweight, minimalist, easy to use and auto-contained, I’m going to use the next:
– Python as lightweight and powerful programming language.
– Flask (http://flask.pocoo.org) is a microframework for Python based on Werkzeug and Jinja 2. I will use Flask to implement a mini-web application.
– SQLAlchemy (http://www.sqlalchemy.org/) is a Python SQL toolkit and ORM.
– SQLite3 (https://www.sqlite.org) is a software library that implements a self-contained, serverless, zero-configuration, transactional SQL database engine.
– pyOpenSSL library to work with X.509 certificates. Required to start the embedded Webserver on HTTPS (TLS).
– CORS extension for Flask (https://flask-cors.readthedocs.org) useful to solve cross-domain Ajax request issues.
This Docker container provides a Microservice (API Rest) to MAC Address Manufacturer resolution. This Docker container is part of the “Everything generates Data: Capturing WIFI Anonymous Traffic using Raspberry Pi and WSO2 BAM” blog serie (Part I, Part II & Part III), but you can use it independently as part of other set of Docker containers.
This Docker Container will work in this scenario, as shown above image. Then, let’s do it.
I. Preparing the Python development environment in Mac OSX
Follow this guide to setup your Python Development Environment in your Mac OSX: https://github.com/chilcano/how-tos/blob/master/Preparing-Python-Dev-Env-Mac-OSX.md
II. Creating a MAC Address Manufacturer DB
Exist in Internet several MAC Address Lookup Tools, in fact, the OUI’s prefix used to identify the MAC Address are public available.
But, in this case I am going to use the MAC Address List of Wireshark (https://www.wireshark.org/tools/oui-lookup.html).
Wireshark is a popular network protocol analyzer a.k.a. network sniffer, the Wireshark tool uses internally the MAC Address list to identity the Manufacturer of a NIC.
I’m going to download and create a API Rest for you. Below the steps.
1) Downloading the Wireshark MAC Addresses Manufacturer file and loading into a DB
Using the below Python script I will download the Wireshark MAC Address list into a file and to get the hash. The idea is to parse the file and load it into a minimalist DB.
I will use SQLite Database where I will create an unique table and all information will be loaded there. The Table structure will be:
mac String # The original MAC Address manuf String # The original Manufacturer name manuf_desc String # The Manufacturer description, if exists.
Here the Python script used to do that: mac_manuf_wireshark_file.py
III. Exposing the MAC Address Manufacturer DB as an API Rest
After creating the database, the next step is to expose the data through a simple API Rest. The idea is to make a call GET to the API Rest with a MAC Address and get the Manufacturer as response.
1) Defining the API
The best way to define an API Rest and the contract is using the Swagger language (http://swagger.io). The idea is to create documentation about the API Rest and explain what resources are available or exposed, writte a request and response sample, etc.
In this scenario I’m going to define in a simple way the API, also I’m going to use JSON to define the request and response.
Then, below the API definition.
POST /chilcano/api/manuf # Add a new Manufacturer
PUT /chilcano/api/manuf # Update an existing Manufacturer
GET /chilcano/api/manuf/{macAddress} # Find Manufacturer by MAC Address
In this Proof-of-Concept I will implement only the GET resource for the API.
2) Implementing the API Rest
I have created 2 Python scripts to implement the API Rest.
The first one (mac_manuf_table_def.py) is just a Model of the MacAddressManuf table.
#!/usr/bin/python
# -*- coding: utf-8 -*-
#
# file name: mac_manuf_table_def.py
#
from sqlalchemy import create_engine, ForeignKey
from sqlalchemy import Column, Date, Integer, String
from sqlalchemy.ext.declarative import declarative_base
engine = create_engine('sqlite:///mymusic.db', echo=True)
Base = declarative_base()
#
# Model for 'MacAddressManuf':
# used for API Rest to get access to data from DB
#
class MacAddressManuf(Base):
""""""
__tablename__ = "MacAddressManuf"
mac = Column(String, primary_key=True)
manuf = Column(String)
manuf_desc = Column(String)
def __init__(self, manuf, manuf_desc):
""""""
self.manuf = manuf
self.manuf_desc = manuf_desc
And second Python script (mac_manuf_api_rest.py) implements the API Rest. You can review the
#!/usr/bin/python
# -*- coding: utf-8 -*-
#
# file name: mac_manuf_api_rest.py
#
import os, re
from flask import Flask, jsonify
from flask.ext.cors import CORS
from sqlalchemy import create_engine
from sqlalchemy.orm import sessionmaker
from mac_manuf_table_def import MacAddressManuf
ROOT_DIR = "manuf"
FINAL_MANUF_DB_FILENAME = "mac_address_manuf.db"
HTTPS_ENABLED = "true"
engine = create_engine("sqlite:///" + os.path.join(ROOT_DIR, FINAL_MANUF_DB_FILENAME))
Session = sessionmaker(bind=engine)
app = Flask(__name__)
cors = CORS(app, resources={r"/chilcano/api/*": {"origins": "*"}})
#
# API Rest:
# i.e. curl -i http://localhost:5000/chilcano/api/manuf/00:50:5a:e5:6e:cf
# i.e. curl -ik https://localhost:5443/chilcano/api/manuf/00:50:5a:e5:6e:cf
#
@app.route("/chilcano/api/manuf/<string:macAddress>", methods=["GET"])
def get_manuf(macAddress):
try:
if re.search(r'^([0-9A-Fa-f]{2}[:-]){5}([0-9A-Fa-f]{2})$', macAddress.strip(), re.I).group():
# expected MAC formats : a1-b2-c3-p4-q5-r6, a1:b2:c3:p4:q5:r6, A1:B2:C3:P4:Q5:R6, A1-B2-C3-P4-Q5-R6
mac1 = macAddress[:2] + ":" + macAddress[3:5] + ":" + macAddress[6:8]
mac2 = macAddress[:2] + "-" + macAddress[3:5] + "-" + macAddress[6:8]
mac3 = mac1.upper()
mac4 = mac2.upper()
session = Session()
result = session.query(MacAddressManuf).filter(MacAddressManuf.mac.in_([mac1, mac2, mac3, mac4])).first()
try:
return jsonify(mac=result.mac, manuf=result.manuf, manuf_desc=result.manuf_desc)
except:
return jsonify(error="The MAC Address '" + macAddress + "' does not exist"), 404
else:
return jsonify(mac=macAddress, manuf="Unknown", manuf_desc="Unknown"), 404
except:
return jsonify(error="The MAC Address '" + macAddress + "' is malformed"), 400
if __name__ == "__main__":
if HTTPS_ENABLED == "true":
# 'adhoc' means auto-generate the certificate and keypair
app.run(host="0.0.0.0", port=5443, ssl_context="adhoc", threaded=True, debug=True)
else:
app.run(host="0.0.0.0", port=5000, threaded=True, debug=True)
This second Python script performs the next tasks:
- Calls the Model (
mac_manuf_table_def.py). - Connects to SQLite Database and creates a Session.
- Runs a query by using
macAddressas parameter. - And creates a JSON response with the query’s result.
3) Running and Testing the API Rest
We could use the Flask buit-in HTTP server just for testing and debugging. To run the above Python Web Application (API Rest) just execute the Python script. Note that actually I have 3 versions (py-1.0, py-1.1 and py-latest)
Chilcano@Pisc0 : ~/1github-repo/docker-mac-address-manuf-lookup/python/1.0 $ python mac_manuf_api_rest.py * Running on http://0.0.0.0:5000/ (Press CTRL+C to quit) * Restarting with stat * Debugger is active! * Debugger pin code: 258-876-642 Chilcano@Pisc0 : ~/1github-repo/docker-mac-address-manuf-lookup/python/1.1 $ python mac_manuf_api_rest.py * Running on https://0.0.0.0:5443/ (Press CTRL+C to quit) * Restarting with stat * Debugger is active! * Debugger pin code: 258-876-642 Chilcano@Pisc0 : ~/1github-repo/docker-mac-address-manuf-lookup/python/latest $ python mac_manuf_api_rest.py * Running on https://0.0.0.0:5443/ (Press CTRL+C to quit) * Restarting with stat * Debugger is active! * Debugger pin code: 258-876-642
Now, from other Terminal call the API Rest using curl, I’m going to use only the python-lates version:
$ curl -ik https://127.0.0.1:5443/chilcano/api/manuf/00-50:Ca-Fe-Ca-Fe
HTTP/1.0 200 OK
Content-Type: application/json
Content-Length: 93
Server: Werkzeug/0.11.4 Python/2.7.11
Date: Thu, 03 Mar 2016 17:37:45 GMT
{
"mac": "00:50:CA",
"manuf": "NetToNet",
"manuf_desc": "# NET TO NET TECHNOLOGIES"
}
$ curl -ik https://127.0.0.1:5443/chilcano/api/manuf/11-50:Ca-Fe-Ca-Fe
HTTP/1.0 404 NOT FOUND
Content-Type: application/json
Content-Length: 67
Server: Werkzeug/0.11.4 Python/2.7.11
Date: Thu, 03 Mar 2016 17:38:49 GMT
{
"error": "The MAC Address '11-50:Ca-Fe-Ca-Fe' does not exist"
}
curl -ik https://127.0.0.1:5443/chilcano/api/manuf/00-50:Ca-Fe-Ca-Fe---
HTTP/1.0 400 BAD REQUEST
Content-Type: application/json
Content-Length: 68
Server: Werkzeug/0.11.4 Python/2.7.11
Date: Thu, 03 Mar 2016 17:39:23 GMT
{
"error": "The MAC Address '00-50:Ca-Fe-Ca-Fe---' is malformed"
}
But if you want to run in Production. In the Flask webpage (http://flask.pocoo.org/docs/0.10/deploying/wsgi-standalone) recommends these HTTP servers (Standalone WSGI Containers):
- Gunicorn
- Tornado
- Gevent
- Twisted Web
IV. Putting everything in a Docker Container
1) The Dockerfile
The latest version of the MAC Address Manufacturer lookup Docker container is the python-latest (aka Docker MAC Manuf) and has the next Dockerfile:
# Dockerfile to MAC Address Manufacturer Lookup container. FROM python:2.7 MAINTAINER Roger CARHUATOCTO <chilcano at intix dot info> RUN pip install --upgrade pip RUN pip install unicodecsv RUN pip install Flask RUN pip install sqlalchemy RUN pip install pyOpenSSL RUN pip install -U flask-cors # Allocate the 5000/5443 to run a HTTP/HTTPS server EXPOSE 5000 5443 COPY mac_manuf_wireshark_file.py / COPY mac_manuf_table_def.py / COPY mac_manuf_api_rest.py / RUN python mac_manuf_wireshark_file.py CMD python mac_manuf_api_rest.py
2) Using the Docker Container
Clone the Github repository and build it.
$ git clone https://github.com/chilcano/docker-mac-address-manuf-lookup.git $ cd docker-mac-address-manuf-lookup $ docker build --rm -t chilcano/mac-manuf:py-latest python/latest/.
Or Pull from Docker Hub.
$ docker login $ docker pull chilcano/mac-manuf-lookup:py-latest $ docker images REPOSITORY TAG IMAGE ID CREATED VIRTUAL SIZE chilcano/mac-manuf-lookup py-latest 19d33a4f3ec1 16 minutes ago 714.8 MB
Run and check the container.
$ docker run -dt --name=mac-manuf-py-latest -p 5443:5443/tcp chilcano/mac-manuf-lookup:py-latest $ docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 4b0bb0b5b518 chilcano/mac-manuf-lookup:py-latest "/bin/sh -c 'python m" 2 minutes ago Up 2 minutes 5000/tcp, 0.0.0.0:5443->5443/tcp mac-manuf-py-latest
Gettting SSH access to the Container to check if SQLite DB exists.
$ docker exec -ti mac-manuf-py-latest bash
Getting the Docker Machine IP Address.
$ docker-machine ls NAME ACTIVE DRIVER STATE URL SWARM ERRORS default * virtualbox Running tcp://192.168.99.100:2376 machine-dev - virtualbox Stopped machine-test - virtualbox Stopped
Testing/Calling the Microservice (API Rest).
$ curl -i http://192.168.99.100:5000/chilcano/api/manuf/00-50:Ca-ca-fe-ca
HTTP/1.0 200 OK
Content-Type: application/json
Content-Length: 93
Server: Werkzeug/0.11.4 Python/2.7.11
Date: Sat, 20 Feb 2016 09:01:38 GMT
{
"mac": "00:50:CA",
"manuf": "NetToNet",
"manuf_desc": "# NET TO NET TECHNOLOGIES"
}
If the embedded server was started on HTTPS, you could test it as shown below.
$ curl -ik https://192.168.99.100:5443/chilcano/api/manuf/00-50:Ca-ca-fe-ca
HTTP/1.0 200 OK
Content-Type: application/json
Content-Length: 93
Server: Werkzeug/0.11.4 Python/2.7.11
Date: Mon, 29 Feb 2016 15:58:21 GMT
{
"mac": "00:50:CA",
"manuf": "NetToNet",
"manuf_desc": "# NET TO NET TECHNOLOGIES"
}
V. And now what?, How to use the MAC Manuf Docker with the WSO2 BAM Docker?
Visualizing Captured WIFI Traffic in Realtime from WSO2 BAM Dashboard
As you can see in above image, when capturing WIFI traffic the information is shown in the WSO2 BAM Dashboard but not the MAC Address Manufaturer.
In this scenario, our Docker MAC Manuf will be useful because It will provide the Manufacturer information via a RESTful Microservice. Then, the idea is configure the WSO2 BAM Dashboard (the prepared Kismet Toolbox) to point to the Docker MAC Manuf RESTful Microservice. In other words, the WSO2 BAM will call to the Docker MAC Manuf Microservice to get the Manufacturer information.
The next blog post I will explain how to connect the MAC Address Manufacturer Docker Container with the WSO2 BAM Docker Container by using Docker Compose to do a minimal orchestration.
VI. Conclusions
Python and a few modules (as Flask, SQLAlchemy, CORS, pyOpenssl, …) more you can create quickly any kind of Applications (Business Applications, Web Applications, Mobile Applications, Microservices, …). The development of this (Micro)service and put It into a Docker container was a smooth experience. It was possible to implement the older scripts to automatize some task while at the same time implement modern layered web applications as a microservice, and everything in a few lines of code.
See you soon.
www.intix.info
Holistic Security's RSS
Roger Carhuatocto
github.com / chilcano
Leave a Reply