This is a continuation of the pre-processing feature from data acquisition. A lot of times the files that we received would contain special characters, or inconsistent schema making it tricky to automate. The feature allows data engineers to explore files by reporting inconsistencies and filtering out abnormal files and then uses GPT to generate python based pre-processors to handle novel use-cases. Once created, these pre-processors help automate parsing of files and reduce manual work for data engineers.