Christopher Talib
b9f0ac688c
Removing extract.go file and refactoring the adhoc functions to files with name of the service
2020-01-29 11:05:05 +01:00
Christopher Talib
67aecd65c1
Saving edges and node from CertStream traffic.
...
This work extracts fingerprints and domains from CertStream data
structure received through the stream. It builds nodes and edges and
saves them to the relevant files. It sends this data to Kafka but no
logic is implemented at the exit of the broker yet.
2020-01-29 10:03:52 +01:00
Christopher Talib
de0c5bfe83
Setting up linking and and creating nodes already from the input source
2020-01-28 23:52:24 +01:00
Christopher Talib
7785372e3a
Refactoring from utils to models
...
This work refactors saving and extracting function from the utils
package to the models package as it is a main component of the tool.
`utils` will take care of not related to models functions (such as
finding the files for example).
Also creating unique files for each type of source we are parsing.
2020-01-28 16:02:17 +01:00
Christopher Talib
01f3e3ca1e
Only saving objects that have an ID
2020-01-23 15:01:08 +01:00
Christopher Talib
2548c19ca4
saving
2020-01-23 13:13:59 +01:00
Christopher Talib
64e1d3d7a4
Cleaning up non used functions
2020-01-22 16:33:07 +01:00
Christopher Talib
0d082374e9
Some temporal refactoring, saving domains and nodes
2020-01-22 15:45:25 +01:00
Christopher Talib
d33b293e7c
Read/write from Kafka, simpler structure with plain data and not pointers
2020-01-22 15:01:07 +01:00
Christopher Talib
48cc976595
Adding a consummer for Kafka, WIP
2020-01-21 16:50:50 +01:00
Christopher Talib
06f42fc472
Moving SendToKafka function to broker package
2020-01-17 14:02:52 +01:00
Christopher Talib
e9c065bcc8
Setting up connector to Kafka and sending consummed and formatted Nodes to Kafka
2020-01-16 15:56:57 +01:00
Christopher Talib
1b25c25aa7
Adding some configuration, flagging inputs as CertStream
2020-01-15 16:46:13 +01:00
Christopher Talib
1081e0c728
Full saving of CertStream data
...
This work saves the entires CertStream into JSON.
2020-01-15 16:07:11 +01:00
Christopher Talib
b7dce16c9e
Adding unpacking in JSON format, and types for certstream format
2020-01-14 16:31:57 +01:00
Christopher Talib
7d741f4cf3
Basic read/write implementation with output in JSON
2020-01-14 14:49:14 +01:00
Christopher Talib
b4e07b254a
First commit with gitignore for the binary and initializing go modules
2020-01-14 09:43:12 +01:00