SIDN Labs open-sources ENTRADA

We’re proud to announce that we are releasing our ENTRADA platform as an open source project. ENTRADA enables researchers and network engineers to analyse large amounts of network traffic, for instance to spot anomalies and threats. We originally developed ENTRADA to analyse DNS traffic and to further increase the security and stability of SIDN’s services for .nl (hence the acronym: “ENhanced Top-level Domain Resilience through Advanced Data Analysis”).

What is ENTRADA?

In a nutshell, ENTRADA is a “big data” platform designed to ingest and quickly analyse large amounts of network data. More technically, it is in fact a high-performance data-streaming warehouse (DSW).ENTRADA's ability to deliver such performance is due to two main features:

Please refer to our research paper for a performance evaluation and more details.

Who can use it?

  • Domain registries that are interested in developing DNS big data applications

  • Internet measurement researchers who are in need of a high-performance analytics platform

Main features

  • Performance: analyse the Parquet equivalent of 50 TB of pcap data in under 3.5  minutes, with a small 6-node cluster (4 data processing nodes).

  • Interface: benefit from easy SQL statements to analyse your data

  • Scalable: just add more nodes for faster processing

  • Built-in conversion of DNS/IP/TCP/UDP/ICMP network data to Parquet data

  • Open-source

.nl ENTRADA deployment

We have been using ENTRADA at SIDN Labs for the past year. It currently runs on a relatively small 6-node cluster (with 4 data-nodes), which contains about 2 years of DNS traffic from the .nl name servers. This equates to over 100 billion queries and responses. Every day, the database grows by a further 400 million queries.We use ENTRADA as an enabling platform to develop applications that aim at increasing the stability and security of the .nl internet zone and the global internet infrastructure. In ENTRADA, we store .nl zone DNS traffic and analyse it to detect anomalies and threats. It has many potential applications: for example, we are working on including algorithms for phishing detection and the identification of botnet traffic (https://irtf.org/raim-2015-slides/ic/moura.pdf)

Open source

We decided to make ENTRADA open source because we believe it might also benefit the larger network analysis community. By releasing ENTRADA as an open source project, we hope to jumpstart a community of ENTRADA developers and users. New applications based on ENTRADA can aid in further enhancing the stability and security of the internet as a whole.If you have any questions about deploying or running ENTRADA at your organisation, send an e-mail to entrada {at} sidnlabs.nl.For more information see the ENTRADA project-site at: http://entrada.sidnlabs.nl/

Comments

Maarten_Wullink_thumb

Maarten Wullink

Research engineer

+31 26 352 55 00

maarten.wullink@sidn.nl

  • Tuesday 28 November 2017

    News

    Is everything on the internet true?

    Thumb-kids-telephone

    School kids aren't as internet-savvy as they like to think

    Read more
  • Thursday 12 October 2017

    Weblog

    Looks too good to be true? Then it probably is

    Thumb-women-online-shopping

    Don't fall for fake webshop scams!

    Read more
  • Tuesday 30 October 2018

    News

    Privacy Portal now open to assist registrars

    Thumb-Europe-GDPR

    A single tool for all your privacy management tasks

    Read more

Sorry

Your browser is too old to optimally experience this website. Upgrade your browser to improve your experience.