Text files
You can process files with the text
format option to parse each line in any text-based file as a row in a DataFrame. This can be useful for a number of operations, including log parsing. It can also be useful if you need to ingest CSV or JSON data as raw strings.
For more information, see text files.
Options
See the following Apache Spark reference articles for supported read and write options.