Christopher Talib
3961e79062
Matcher logic and IOCs
...
This work starts to build the matcher logic into styx. For the moment,
the goal is to define IOCs and load them when the Matcher plugin is
activated.
To implement: Then, the matcher will run periodic queries to different
types of nodes and index them to its one Matcher Dgraph Node. So be
targetting a specific IOCs, the user will be able to list the
observation that have been made to it.
2020-05-29 11:32:55 +02:00
Christopher Talib
9547aeea3f
basic matching on simple target, no sending to dgraph
2020-05-27 12:05:53 +02:00
Christopher Talib
9fa5d13bf6
Full text search and indexing some keywords
...
Some of keywords are indexed and open for full text search, please refer
to the README for more details.
CertStream, Pastebin and Shodan are running as services and can be
searched.
Next steps: building the matcher and creating edges.
2020-05-20 10:03:28 +02:00
Christopher Talib
7163147a4f
Pastebin nodes simple
...
Pastebin data is also sent to Dgraph and can be queried.
2020-05-19 10:10:42 +02:00
Christopher Talib
b1ca4b3c5f
Shodan in Dgraph, first part
...
Implementing first version for shodan node, missing yet some models, but
the overal approach works and can be queried in Ratel.
2020-05-18 16:09:04 +02:00
Christopher Talib
cbdca52ab2
Simple linked model on certstream + better install instructions
2020-05-18 10:22:08 +02:00
Christopher Talib
5dca0a0472
adding Dgraph explanantions
2020-05-13 14:52:38 +02:00
Christopher Talib
f0aa10a9d2
Adding docker-compose for dgraph
2020-05-13 11:51:54 +02:00
Christopher Talib
d0c8deae99
saving
2020-03-19 09:27:15 +01:00
Christopher Talib
fb270a1b66
Mutating data without needing to get it back
2020-03-04 16:34:14 +01:00
Christopher Talib
f61fe566a5
Basic connection to Dgraph DB
...
The first work and input to the graph db is set up in this work. It's
for the moment very basic and doesn't cover relations and only works for
certstream data.
2020-03-04 15:16:59 +01:00
Christopher Talib
b72e82071d
saving
2020-03-02 17:06:28 +01:00
Christopher Talib
cd43194873
First work for the dgraph connection + fixing some error logging
2020-03-02 16:27:51 +01:00
Christopher Talib
d761e824f3
Changing the plugin architecture for a modular architecture.
2020-02-25 10:05:31 +01:00
Christopher Talib
b2da64a9d7
Enh/modular arch
2020-02-25 10:05:31 +01:00
Christopher Talib
e87856b9a1
Removing print statements
2020-02-20 14:53:18 +01:00
Christopher Talib
0f25d6d81c
Prototype to domain filtering with Pharos filters
2020-02-20 14:32:06 +01:00
Christopher Talib
e7421931c2
Couple of quickfixes to allow run a dry installion (deactivating ES and balboa)
2020-02-19 15:26:47 +01:00
Christopher Talib
56e0e52bb5
Cleaning up example config
2020-02-19 10:03:49 +01:00
Christopher Talib
5bd0b8090b
Update dependencies
2020-02-18 12:22:23 +01:00
Christopher Talib
3a3637c847
Move RunIPFilters() to filters/main.go
2020-02-18 12:21:39 +01:00
Christopher Talib
f0a6715acd
Adding elasticsearch configuration on README
2020-02-17 12:10:27 +01:00
Christopher Talib
f4f82d9e1c
Adding code and setting up elastic search
2020-02-17 12:08:49 +01:00
Christopher Talib
5b1bfbc195
Filters for IP and shodan
2020-02-14 20:46:09 +01:00
Christopher Talib
1fc11e7a4e
Adding all CIDR files in filters/data
2020-02-14 12:01:50 +01:00
Christopher Talib
02a014262a
Adding Akamai filters for data recieved in Shodan + allow deactivating Balboa search
2020-02-14 11:36:47 +01:00
Christopher Talib
e963633c89
Allowing the possiblity to activate or deactivate modules
2020-02-14 11:30:59 +01:00
Christopher Talib
bd3f108d12
Shodan ports in config
2020-02-12 16:54:14 +01:00
Christopher Talib
638216e8fa
Implementing config variables in the application
2020-02-10 16:11:25 +01:00
Christopher Talib
7c5b2b714a
Adding configuration documentation and the config file in the gitignore
2020-02-10 14:40:33 +01:00
Christopher Talib
e634636768
Fix readme
2020-02-10 10:36:36 +01:00
Christopher Talib
40bfc4b01d
Beautify readme
2020-02-07 17:50:07 +01:00
Christopher Talib
2991f830bb
Update README with info about Balboa queries
2020-02-07 17:45:37 +01:00
Christopher Talib
afd99c0a4d
Adding balboa enrichment for domains and hostnames + documentation
2020-02-07 17:39:33 +01:00
Christopher Talib
787e2c3d02
Creating balboa package
2020-02-07 17:38:43 +01:00
Christopher Talib
01f7ae70b9
Renaming CertStreamWrapper to CertStreamRaw for more clarity
2020-02-07 15:50:21 +01:00
Christopher Talib
e38b05de66
Update README with more information on the nodes and edges connections
2020-02-07 15:49:42 +01:00
Christopher Talib
c7a52c527a
Refactoring saving single value into a helper, adding meta names to main nodes and edges
2020-02-07 15:27:14 +01:00
Christopher Talib
93f577cae9
Settting up more verbose logging with calling function and timestamp
2020-02-07 14:53:39 +01:00
Christopher Talib
2813ab0e96
Cleaning up and and reorganizing the models package
2020-02-06 17:16:47 +01:00
Christopher Talib
cc87dab922
Cleaning up and and reorganizing the models package
2020-02-06 17:13:19 +01:00
Christopher Talib
b5caae0974
Shodan connector and saving up and running
2020-02-06 17:01:37 +01:00
Christopher Talib
a321b46d0f
Adding chained items to parent object in certstream
2020-02-05 15:57:30 +01:00
Christopher Talib
c2b59808be
First implementation of shodan connector, but error on API key on streaming
2020-02-05 14:46:52 +01:00
Christopher Talib
efde1a4a54
Building ID on certnode
2020-02-04 10:58:56 +01:00
Christopher Talib
380e51229b
Building nodes, certnode, and edges
2020-02-04 10:57:14 +01:00
Christopher Talib
00bfb5eaaf
First implementation of certnode
2020-02-04 10:53:09 +01:00
Christopher Talib
f0f6457dd3
go mod tidy
2020-02-04 10:35:39 +01:00
Christopher Talib
8fc88b2694
Merge branch 'pastebin-poc' into 'master'
...
Working pastebin connector with parallel running with certstream
See merge request LABS/styx!1
2020-02-03 10:02:43 +01:00
Christopher Talib
0cd0faa6fa
Working pastebin connector with parallel running with certstream
...
Parallel running of Certstream and Pastebin connectors in go routine
with wait groups to exit them properly.
2020-02-03 10:01:52 +01:00
