Commit graph

13 commits

Author SHA1 Message Date
Christopher Talib 3961e79062 Matcher logic and IOCs
This work starts to build the matcher logic into styx. For the moment,
the goal is to define IOCs and load them when the Matcher plugin is
activated.

To implement: Then, the matcher will run periodic queries to different
types of nodes and index them to its one Matcher Dgraph Node. So be
targetting a specific IOCs, the user will be able to list the
observation that have been made to it.
2020-05-29 11:32:55 +02:00
Christopher Talib b2da64a9d7 Enh/modular arch 2020-02-25 10:05:31 +01:00
Christopher Talib 02a014262a Adding Akamai filters for data recieved in Shodan + allow deactivating Balboa search 2020-02-14 11:36:47 +01:00
Christopher Talib e458a327d6 Moving SaveDomains to parser package 2020-01-28 16:05:36 +01:00
Christopher Talib 7785372e3a Refactoring from utils to models
This work refactors saving and extracting function from the utils
package to the models package as it is a main component of the tool.
`utils` will take care of not related to models functions (such as
finding the files for example).

Also creating unique files for each type of source we are parsing.
2020-01-28 16:02:17 +01:00
Christopher Talib e4cc92e100 First work on Pastebin parser 2020-01-26 17:27:20 +01:00
Christopher Talib 2548c19ca4 saving 2020-01-23 13:13:59 +01:00
Christopher Talib 8ac9d5786b Refactoring and setting clearer filenames in utils/ 2020-01-22 15:49:52 +01:00
Christopher Talib 0d082374e9 Some temporal refactoring, saving domains and nodes 2020-01-22 15:45:25 +01:00
Christopher Talib d33b293e7c Read/write from Kafka, simpler structure with plain data and not pointers 2020-01-22 15:01:07 +01:00
Christopher Talib 1081e0c728 Full saving of CertStream data
This work saves the entires CertStream into JSON.
2020-01-15 16:07:11 +01:00
Christopher Talib 6eaae99668 Extracting structures from CertStream
This work builds an extractor for the data in the CertStream in order to
save it. It builds itself from the previous work, so extensions and
flags can be added to the structures. The work in `utils` is basically a
big extractor for the data taking advantage of the JSONq library.

Currently, there is not refactoring and the "chains" are not saved
because they need additionnal computation which will come in a later
commit.
2020-01-15 14:36:53 +01:00
Christopher Talib b7dce16c9e Adding unpacking in JSON format, and types for certstream format 2020-01-14 16:31:57 +01:00