ndudaev/nginx-parser
Watch
1
0
Fork
You've already forked nginx-parser
 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

add enrichment functions, that add new calculated fields to the database; read CHANGELOG.md for details #1

Merged
ndudaev merged 1 commit from dev-add-error-logs into main 2026-10-07 13:49:41 +00:00
Conversation 0 Commits 1 Files changed 8 +255 -2
ndudaev commented 2026-10-07 13:49:22 +00:00
Owner
Copy link

Additional Fields

2026-10-07 - add UAClassifier to spec.go with regex for common probes and bots
2026-10-07 - add ExtractMethod to utils.go. Function extracts method name from request field of the logs
2026-10-07 - add PopulateMethod to enrich.go. Function creates new field in the logs with method name extracted
2026-10-07 - add PopulateIsGooglebot to enrich.go. Function checks if "Googlebot" string is in UA
2026-10-07 - add PopulateIsToolUA to enrich.go. Function checks against regex that captures typical tools names in UA
2026-10-07 - add PopulateIsProbe to enrich.go. Function checks against regex that captures requests for various files, dirs and their combinations
2026-10-07 - add PopulateUAClass to enrich.go. Function looks at different types of UA's and assigns them to a class of UA
2026-10-07 - add PopulateUAFamily to enrich.go. Function attempts to identify a few UA families depending on the content of the UA string
2026-10-07 - add PopulateProbeGroup to enrich.go. Function attempts to identify different probe groups based on the content of the UA string

## Additional Fields `2026-10-07` - add `UAClassifier` to `spec.go` with regex for common probes and bots `2026-10-07` - add `ExtractMethod` to `utils.go`. Function extracts method name from `request` field of the logs `2026-10-07` - add `PopulateMethod` to `enrich.go`. Function creates new field in the logs with method name extracted `2026-10-07` - add `PopulateIsGooglebot` to `enrich.go`. Function checks if "Googlebot" string is in UA `2026-10-07` - add `PopulateIsToolUA` to `enrich.go`. Function checks against regex that captures typical tools names in UA `2026-10-07` - add `PopulateIsProbe` to `enrich.go`. Function checks against regex that captures requests for various files, dirs and their combinations `2026-10-07` - add `PopulateUAClass` to `enrich.go`. Function looks at different types of UA's and assigns them to a class of UA `2026-10-07` - add `PopulateUAFamily` to `enrich.go`. Function attempts to identify a few UA families depending on the content of the UA string `2026-10-07` - add `PopulateProbeGroup` to `enrich.go`. Function attempts to identify different probe groups based on the content of the UA string
ndudaev deleted branch dev-add-error-logs 2026-10-07 13:49:42 +00:00
Sign in to join this conversation.
Reviewers
No reviewers
Labels
Clear labels
No items
No labels
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
ndudaev/nginx-parser!1
No description provided.