Commit graph

12 commits

Author SHA1 Message Date
Christopher Talib
668686fbab Saving edges and node from CertStream traffic.
This work extracts fingerprints and domains from CertStream data
structure received through the stream. It builds nodes and edges and
saves them to the relevant files. It sends this data to Kafka but no
logic is implemented at the exit of the broker yet.
2020-01-29 11:02:19 +01:00
Christopher Talib
67aecd65c1 Saving edges and node from CertStream traffic.
This work extracts fingerprints and domains from CertStream data
structure received through the stream. It builds nodes and edges and
saves them to the relevant files. It sends this data to Kafka but no
logic is implemented at the exit of the broker yet.
2020-01-29 10:03:52 +01:00
Christopher Talib
de0c5bfe83 Setting up linking and and creating nodes already from the input source 2020-01-28 23:52:24 +01:00
Christopher Talib
7785372e3a Refactoring from utils to models
This work refactors saving and extracting function from the utils
package to the models package as it is a main component of the tool.
`utils` will take care of not related to models functions (such as
finding the files for example).

Also creating unique files for each type of source we are parsing.
2020-01-28 16:02:17 +01:00
Christopher Talib
a0d2761c4e Adding RFC3339 times 2020-01-26 17:26:03 +01:00
Christopher Talib
d33b293e7c Read/write from Kafka, simpler structure with plain data and not pointers 2020-01-22 15:01:07 +01:00
Christopher Talib
48cc976595 Adding a consummer for Kafka, WIP 2020-01-21 16:50:50 +01:00
Christopher Talib
06f42fc472 Moving SendToKafka function to broker package 2020-01-17 14:02:52 +01:00
Christopher Talib
e9c065bcc8 Setting up connector to Kafka and sending consummed and formatted Nodes to Kafka 2020-01-16 15:56:57 +01:00
Christopher Talib
e6cbfc6466 Changing IDs to stix compatible format 2020-01-16 12:06:03 +01:00
Christopher Talib
b7dce16c9e Adding unpacking in JSON format, and types for certstream format 2020-01-14 16:31:57 +01:00
Christopher Talib
7d741f4cf3 Basic read/write implementation with output in JSON 2020-01-14 14:49:14 +01:00