Why exporting to CSV is not so easy?

Dmitry Ch
3 min readMar 27, 2023

--

Exporting data in CSV format, what could be easier?

We take the data and split it into columns, choosing a semicolon or a comma as a delimiter… umm… or maybe a comma?

Okay, what about line breaks? Will it open correctly on MacOS? And what about Google Docs?
It turns out that not everything is as simple and straightforward as it seems.

It’s not rocket science, but there are still things to consider.
So, let’s dive in.

Delimiters

How will our data be split into columns? Obviously, with the help of a delimiter character. But which one? Is there a standard value for it? No, there isn’t.
Possible options include:
• Comma
• Semicolon
• Tab
• Space

Okay, what about the “default delimiter”?

Well, it also depends. Let’s take Microsoft Excel as an example.

In the Windows version, the default delimiter is a comma. But in Microsoft Excel for MacOS, it’s already a semicolon.

Thank you, Microsoft :/

Oh, and don’t forget that all this works only if you specify the “UTF-8-with-bom” “\ufeff” encoding identifier at the beginning of the file content.

So, what about Google Docs then?
The folks at Google are amazing guys. They handle both commas and semicolons perfectly.
In reality, the delimiter is just one of the parameters that can differ in various software versions for viewing CSV files.

In addition to it, there are:
1. Line break delimiter
2. Escape char
3. Quote char
4. Double quote
5. Encoding
6. Quoting type

There are a thousand of them… And how is the user supposed to know all of them to generate a valid CSV document?

At hexway, we always strive to think about the user experience when working with our tools (just a friendly reminder that we have an excellent platform for vulnerability management & pentest automation). And the ability to export vulnerabilities found in CSV format is one of the features that we couldn’t miss. But to make it easier for the user, we have pre-set export settings profiles, so that you wouldn’t have to keep in mind which delimiter to specify.

CSV export

Of course, we also left the possibility of manual export settings.

As you can see, even such simple tasks as CSV exporting require a certain amount of analysis and testing. Maybe we didn’t manage to cover all possible CSV settings (such as date formats or delimiter dependencies on language settings), but the chosen export profile scheme allows us to easily add them.

And as you may already noticed, in hexway Apiary, there is a convenient feature for exporting to CSV (as well as project grouping and applying these groups in dashboard filters, and a bunch of other new features).

Be sure to try it out!

--

--

Dmitry Ch
Dmitry Ch

Written by Dmitry Ch

Information security researcher, entrepreneur, speaker. https://hexway.io

No responses yet