File System (CSV /Flatfile) to DB (MySQL)

 In this example, I will explain how to process FlatFiles to DB

What is a Flat File?

Flat files present complex hierarchical structural data in a record–based storage format. Unlike XML, flat files do not embed structural data (metadata) within the data. The data in the flat file has been “flattened” by removing the hierarchical relationship between records, leaving the records intact as a single logical record of application data.

All flat files consist of a list of records containing fields and composites:

• Fields are atomic pieces of data (for example, ID and ID qualifier, Date and time).
• Composites contain multiple fields (for example, ID and ID qualifier, Date and time). The fields within a composite are referred to as subfields.
• Records (also known as segments) are sequences of fields and/or composites.

Flat file Schema:

A flat file schema is a blueprint that contains the instructions for parsing or creating a flat-file. The schema exists as a namespace element in the webMethods Integration Server. This blueprint details the structure of the document, including delimiters, records, and repeated record structures. 

Sample:

EMP,EMPID,ENAME,ESAL

EMP,401,Kiran,10000

EMP,402,Jai,20000

EMP,403,Joseph,30000

EMP,404,Mohan,40000

EMP,405,Swapna,50000

Steps:

1. Create flat file schema as per your requirement. 

    Flat file definition: Specify Record parser type, Record and Fields ..etc.

    File file structure : Define record and field definitions

    Test the schema with your test data. isValid is important field

2. Use convertToValues (WmFlatFile) for converting flat files to IS document Types

3. Use Loop flow stop to process the parsed records to DB

Schema:

Flow Implementation

DB:

GitHub:Source code

Source:

https://documentation.softwareag.com/