Input manager on large files does not suppress a large quantity of errors #3692
Labels
Area: Input
Implementation: Core
Implementation requires modification of the Zeek core
Type: Bug 🐛
Unexpected behavior or output.
When a malformed large input file is loaded into Zeek, I'd expect the framework to limit the errors encountered so to not flood output, based on the info found under: https://docs.zeek.org/en/master/frameworks/input.html#broken-input-data
This doesn't seem to be the case, as the reporter.log file fills up with every error encountered while loading. On a table file containing almost a million entries, most of them faulty, an error is reported for each case resulting in a massive flood.
A simple reproduction can be shown, with an input file generated with:
(echo '#fields\tindicator\tindicator_type\tmeta.source'; yes hello|head -n 1000) > intel.dat
and a zeek script loading it:
cat intel.zeek redef Intel::read_files += { fmt("%s/intel.dat", @DIR) };
Running this script results in a flood of output:
I've attached the two sample intel files here, along with an additional table.zeek which calls the input framework directly to load a table on the intel.dat file. Both zeek scripts result in a reporter.log containing every error encountered.
testFiles.zip
The text was updated successfully, but these errors were encountered: