What are BOM signatures?

What are BOM signatures?

What are BOM signatures?

A Byte Order Mark (BOM) is a special sequence of bytes added at the beginning of a text file to indicate its encoding format, including a UTF-8, UTF-16, or UTF-32. BOM signatures are harmless in regular text files but problematic in security or executable files when a program reads them.
  1. A BOM is an invisible marker placed at the start of a file that, while not visible in normal text, can affect file behavior.
  2. Its purpose is to define how characters are represented internally, ensuring proper text display across platforms.
  3. Some editors (like Notepad or Visual Studio Code) automatically add a BOM when saving a file in UTF-8 with BOM encoding.
For example:
A .pem certificate or .sh script starting with a BOM may trigger errors like:
Info
invalid certificate format
/bin/bash^M: bad interpreter

BOM could be a problem in security.txt and other security files

BOM, when not explicitly required by the file’s specification (such as an RFC), some clients may interpret it as an actual character. This can lead to parsing errors or cause certain fields in the file to be misread, especially by automated systems that process security.txt files.

If security.txt is meant for humans, why does BOM handling matter?

While security.txt was initially intended for humans to identify and report security issues, it’s now also consumed by automated programs. To ensure consistent interpretation and avoid errors, it’s important to follow the RFC strictly and avoid including a BOM unless it’s specifically required.

What are the common BOM signatures?

Some of the common BOM signatures in configuration, log, or script files are:
  1. UTF-8: EF BB BF
  2. UTF-16 LE: FF FE
  3. UTF-16 BE: FE FF

How to detect BOM signatures?

To detect BOMs:
  1. Use any hex editor or command-line tools like:
    1. xxd filename.txt | head -1
    2. hexdump -C filename.txt | head -1
  2. If you see bytes like EF BB BF at the start, that means the file contains a BOM.
  3. In rare cases, if a program doesn’t handle encoding properly, the BOM might show up as strange characters.

What can you do?

  1. Always save configuration or security files without a BOM (choose UTF-8 without BOM).
  2. Use a plain-text editor that allows you to control encoding formats.
  3. If a file already contains a BOM, reopen it in a compatible editor, change encoding to UTF-8 (no BOM), and resave it.