What’s New in Aid4Mail 5?

Version 5.0.21 (released: March 3, 2023)

* Adds News screen to notify you of updates, issues and other news. Currently only available when you close your project (File > Close project).

* Significantly improves processing speed when converting OLM, OST and PST files without Outlook.

* Fixes issues relating to Proxy settings.

* Minor fixes and improvements.

* Updates third-party utilities.

* Updates the User Guide.

Download Aid4Mail 5

Version 5.0.20 (released: February 14, 2023)

* Adds a new setting under App Settings > Advanced > Logging level. When you check the box for “Record minor errors”, Aid4Mail will log warnings and recoverable errors that affect minor elements of email processing. These do not cause a conversion to fail.

* Improves responsiveness of Pause, Stop and Force stop commands.

* Fixes a new MAPI system error (0x800CCE1F) that is caused by recent Outlook updates under Windows 11.

* Other minor fixes and improvements.

Version 5.0.19 (released: December 14, 2022)

* New behavior when “Remove journaling envelope” option is set:

Sender and Recipient emails, which may be missing from the original SMTP header, are stored in the body of the journaling envelope. Aid4Mail now extracts them so that they are included when the search scope is set to All, Recipients and Participants. Those email addresses are also included when using the {EmailAccount} template token. Furthermore, when using the “Add extra metadata to email headers” option, recipient and sender email addresses from the journaling envelope are preserved in the target header through the new X-Journal-Recipients and X-Journal-Sender header fields respectively.

* Adds “Open store on remote server” option for Outlook profile source format. When checked, Aid4Mail bypasses locally downloaded mail (stored in OST files) and collects mail directly from the remote account (i.e. Exchange or IMAP server). This option affects which hidden folders are shown by the “Include hidden system folders”. Note that collecting mail from a remote server is a lot slower than from a local OST file. However, an OST file may not reflect the contents of the account on the server, either because it hasn’t been synchronized or because it’s configured to purge older mail.

* Adds “MapiCategories” search operator to email section. Use it to search category keywords assigned to emails in Outlook. This feature works with the following source formats: Outlook profiles, PST and MSG files with the “Process using Outlook” option checked. Example:

MapiCategories:”Red category”

* Adds workaround to a Microsoft MAPI bug when collecting mail from an Exchange server and converting it to a non-MAPI format.

* Improves organization of user interface menus and main screens.

* Improves incremental processing for OST and PST files processed without Outlook.

* Fixes a regression that caused incremental processing to fail with mbox, Google TakeOut and Vault source files.

* Fixes a regression that saved files as regular PDFs when they were configured as PDF/A .

* Many minor fixes and other improvements.

* Updates third-party utilities.

* Updates the User Guide.

Version 5.0.18 (released: October 11, 2022)

* Adds OAuth2 authentication for Microsoft 365 accounts accessed through the IMAP protocol. Note that this will only work if Multi-Factor Authentication is turned on for your Microsoft account.

* Adds workaround for downloading mail from EarthLink accounts.

* Adds “Export to a single file” option for the CSV and TSV formats.

* Several minor fixes and other improvements.

Version 5.0.17 (released: August 8, 2022)

* Improves loading speed of projects that have many sessions.

* Improves item transfers between source and target MAPI formats (PST, MSG and profiles using Outlook). You should no longer get IConverterSession errors if all the following conditions are met: Aid4Mail license is activated, “Add extra metadata to email headers” is unchecked, “Item filtering” is NOT set to “Search query”, and “Python filtering script” is set to (None).

* Updates .NET framework used by GUI and bridge consoles to version 4.8.

* Removes the Gmail delegate access option due to lack of support by the Google API.

* Several minor fixes and other improvements.

Version 5.0.16 (released: July 1, 2022)

* Adds support for the new Google Vault mbox-based export format.

* Adds a workaround when using native date filtering on a MAPI profile connected to an Exchange server. Prior to this fix, only the first 4096 items matching a criteria in a folder were processed.

* Adds a workaround when using native date filtering through the Graph API (Microsoft 365). Prior to this fix, only the first 275 items matching a criteria in a folder were processed. This is due to an undocumented limitation in Microsoft’s KQL search feature. We recommend you use the OData filtering feature instead.

* Adds a workaround when using Windows PowerShell to execute Aid4Mail Console. PowerShell doesn’t support command-line switches that contain a dot (.), like -Source.Format=EML. You can now replace the dot with a plus sign (+) and Aid4Mail Console will run as designed. Example:

a4m-32.exe -cli -Source+Format=EML -Source+RootLocation=D:\Email\Source -Target+Format=mbox -Target+RootLocation=D:\Email\Target

* Fixes an issue when decoding quoted-printable content in EML or mbox files originating from an Apple computer. When converted to PDF, HTML, or plain text, message text would show “=” signs interspersed in the message text.

* Fixes an issue displaying folders in certain PST files when accessing the contents without Microsoft Outlook.

* Fixes a target PST file issue when appending emails without Microsoft Outlook.

* Fixes an issue when redirecting the output from Aid4Mail Console to a file.

* Updates IMAP server names for Yahoo! Mail and AOL when using autocomplete feature in the IMAP configuration editor. Since May 2022, Yahoo! Mail and AOL limit the collection of emails to 10,000 items per folder and 1000 items when using native searching. To overcome these limitations, you must use their new IMAP server name.

For Yahoo! Mail, replace

imap.mail.yahoo.com

with

export.imap.mail.yahoo.com

For AOL, replace

imap.aol.com

with

export.imap.aol.com

* Updates regular expressions PCRE2 engine to version 10.40.

* Updates third-party utilities.

* Updates the User Guide.

* Several other minor fixes and improvements.

Version 5.0.15 (released: April 8, 2022)

* Adds Source > Format > Email file carving.

Aid4Mail’s new carving feature can extract MIME emails from all kinds of files, including:

    – Uncompressed disk images (DD, E01, AFF, …)

    – Forensically extracted unallocated disk space

    – Natively unsupported mailbox formats like FoxMail, Incredimail and Juno

    – Corrupted mail files

The carving algorithm works without relying on a file’s internal format, such as a disk image Master File Table (MFT) or data placement markers. The only requirements for successful retrieval are:

    – The email is in MIME format (even if incomplete)

    – Its MIME parts are stored contiguously and sequentially

    – It is not stored in a compressed or encrypted container

As a result, carving won’t work with Eudora, Outlook Express 5, PST, OST and OLM files because emails are not stored according to the above requirements.

File carving is a slow and processor intensive task. If you need to carve emails from a large forensic disk image, consider using this feature only on the unallocated space. Most disk imaging tools let you extract and save the unallocated space to a file. Use that file as your carving source in Aid4Mail.

Whenever available, always prefer the native source format to process your files. Aid4Mail is not only faster, but also better able to determine folder names and status information when it handles a known format.

Note that this feature is only available with the Investigator and Enterprise licenses.

* Several minor fixes and improvements.

Version 5.0.14 RC5 (released: February 22, 2022)

* Adds drag-and-drop reordering to session panels.

* Adds a progress indicator in the Aid4Mail Windows taskbar button.

* Adds support for alternative words/terms within a search-list line. The vertical tab “|” symbol is used as a separator. Individual terms can include wildcards. Example:

money|hard cash|dough|moola*|loot

You can also group alternatives when combining with NEAR operators. Examples:

money|(hard<2>cash)|dough|moola*|loot

hidden<5>(money|hard cash|dough|moola*|loot)

* Improves the mbox parser’s accuracy, especially when dealing with corrupted data. The parser is used to extract emails from Thunderbird, Google Takeout, and generic mbox files. It is now able to handle a large variety of mbox and MIME format errors within a mailbox file. As a result, Aid4Mail can find emails that other products either miss or incorrectly merge with the preceding message.

* Improves the handling of partly corrupted OLM files.

* Changes native date filtering workaround when processing Yahoo! Mail and AOL accounts. Due to a bug in the IMAP server software used by those services, the following operators produce unpredictable results:

SENTBEFORE

SENTSINCE

And are substituted with the following when using the “Date selection” filter in Aid4Mail:

BEFORE

SINCE

Example:

This fails with Yahoo! Mail and AOL accounts:

SENTSINCE 01-Jan-2022 SENTBEFORE 26-Jan-2022

This succeeds with the workaround:

SINCE 01-Jan-2022 BEFORE 26-Jan-2022

This workaround uses the IMAP internal date instead of the email’s Sent date. Usually these two dates are similar, but sometimes can differ significantly. As a result, you may miss emails that were sent within the specified date range. This issue, which has existed for a long time, can only be resolved if Yahoo and AOL decide to fix it one day.

* Fixes an IMAP login issue with Rackspace when passwords contain certain special characters.

* Fixes a filtering issue that occurred when the tokenization/stemming options were combined with the “known variation” and “proximity” wildcards.

* Fixes a regression in the mbox parser that, on rare occasions, made it fail to identify emails in some files.

* Many other improvements in both the GUI and engine, as well as some bug fixes.

* Updates regular expressions PCRE2 engine to version 10.00.

* Updates the User Guide.

Version 5.0.12 RC4 (released: December 22, 2021)

Please note that most of the features mentioned below are only available in Aid4Mail Investigator and Enterprise. However, we have improved PDF creation and Proxy options as well as many tweaks and fixes that also benefit Aid4Mail Converter.

* Adds major improvements to native filtering capabilities. The settings are now grouped under their own “Native filtering” section and can be combined with folder filtering. You can filter by date range without having to write a native search query – simply enter the year (yyyy), or year-month (yyyy-mm), or full date (yyyy-mm-dd) in the corresponding fields.

• Note that if you configure a date selection and then click on the “Search query” button, the corresponding date range query will be inserted into the script editor if it is initially empty.

* Adds native filtering to source IMAP formats. See Help for the IMAP query syntax. Note that when you connect to a source Gmail account, you must use Google’s search syntax for Gmail instead of the IMAP syntax.

* Adds native date-range filtering to source MAPI formats: Outlook profiles and PST files.

* Adds settings to the “Basic search” capabilities for item filtering. These now make it possible to configure very effective filters without having to write a search query. Use a search list to filter emails based on the presence or absence of keywords. Furthermore. local mail formats can be filtered by date range – simply enter the year (yyyy), or year-month (yyyy-mm), or full date (yyyy-mm-dd) in the corresponding fields.

* Adds an option to tokenize words in search queries and search lists (Project > Filters). Use this feature to define misspellings, spelling alternatives and synonyms for important words in your filter criteria. See Help for details.

A well-defined word alternatives list offers many benefits, like:

1. Saving time writing search queries and search lists.
2. Significantly shorten and simplify your filter search lists and queries.
3. Reducing the risk of missing responsive documents.

Aid4Mail includes several useful tokenization files. You can use these to add relevant word variations to your own tokenization file. You will find them under the Aid4Mail5\Dictionaries subfolder:

• English misspellings.tkn
• English spelling variations.tkn
• Numbers.tkn
• Roman Numerals.tkn
• US-UK spelling differences.tkn

* Adds an editor for creating and editing search lists, tokenization files and Python scripts.

* Adds a project option to set the default stemming dictionary for item search queries and search lists. Aid4Mail includes an English stemming dictionary, but you can download additional dictionaries for free from our website.

• French: https://www.aid4mail.com/download/stemming/French.zip
• German: https://www.aid4mail.com/download/stemming/German.zip
• Italian: https://www.aid4mail.com/download/stemming/Italian.zip
• Portuguese: https://www.aid4mail.com/download/stemming/Portuguese.zip
• Russian: https://www.aid4mail.com/download/stemming/Russian.zip
• Spanish: https://www.aid4mail.com/download/stemming/Spanish.zip
• Swedish: https://www.aid4mail.com/download/stemming/Swedish.zip

* Adds an option to control the behavior of the {EmailAccount} template token:

-Options > Filters > “Only include emails belonging to corresponding accounts”. When checked, Aid4Mail reduces email duplicates by only including messages that match the target account (assuming the SMTP header provides sufficient information to determine the sender and actual message recipient).

* Adds the ReceivedBy search operator. Use it to find messages delivered to a given email address. Aid4Mail gets this address by searching for SMTP header fields typically used to identify the actual message recipient. In some cases this address cannot be determined. Query example:

• ReceivedBy:joe@aid4mail.com

* Adds the {ReceivedBy} token, which corresponds to the email address of the actual recipient of the message being processed. If the address is not found, the token returns a blank value.

* Adds an option to embed attachments in target PDFs.

* Provides a workaround with PDF links to external attachments. Prior to this fix, links to attachments were broken when a PDF was moved along with its attachments to a different location.

* Adds an option to choose whether to prioritize Sent or Received dates when setting the email’s display date. See Options > Advanced > Email display date

* Adds SOCKS4, SOCKS5 and Tunnel protocols to IMAP Proxy settings.

* Improves the behavior of the * and ? wildcard characters so that they also match themselves. This ensures that if you meant to use those characters as literals and didn’t realize the query is treated as a wildcard pattern, your search will nevertheless match the corresponding phrase.

* Many other improvements in both the GUI and engine, as well as some bug fixes.

* Updates the User Guide.

Version 5.0.11 RC3 (released: September 22, 2021)

* Adds fields for Custodian information in Case screen. In Aid4Mail Investigator and Enterprise, {CustodianName} and {CustodianInitials} template tokens are available for configuring Bates stamps and target folder/file names.

* Adds “Date” search operator. This is based on the email’s display date. Aid4Mail will determine its value by searching the metadata in the following order: Received date, Sent date, Saved date, Modified date, and finally Created date. This makes Date the most trustworthy date operator to use. Example: Date>=2021-09-01

* Adds support for PDF creation using full available RAM on a 64-bit Windows system. To benefit from this feature, please download the following package and unzip in your Aid4Mail5 program folder:

https://www.aid4mail.com/download/PDF-64.zip

* Adds new character wildcards to match letters with or without diacritics as well as ligatures and the typographic apostrophe. For example: na<i>ve will match both naive and naïve; L<ae>titia will match both Laetitia and Lætitia; can<‘>t will match both can’t and can’t. See the “Known variation wildcards” topic in Help for syntax details.

* Adds new word wildcard: <?>. Use it to match any group of characters that don’t contain any white space. For example:

gained <?> today

will match:

gained +$100,000.00 today

* Changes detection of wildcard pattern in filter scripts. When creating a query without specifying a modifier (like L or W), Aid4Mail treated a search term as a wildcard only if it contained at least one “*” character. Now, the presence of any of the following characters will signify a wildcard pattern: * ? # ~

* Changes proximity searching wildcards <n>, <.> and <*> to match words in forward and reverse order. Add “+” to enforce one-way forward matching:

• cat<+12>dog
• cat<+*>dog
• cat<+.>dog

* Removes the “HTML entity” wildcards. The “&” symbol is now treated as a regular character. The “#” symbol has been reassigned to match zero or one non-word character. Examples:

• U#S#A# matches “U.S.A.” and “USA”
• can#t matches “can’t”, “can’t” and “cant”
• #123 matches “#123”, “123”, “-123” and “+123”
• e#discovery matches “ediscovery” and “e-discovery”
• #541##754#3010 matches “(541) 754-3010”, “541-754-3010”, “541-754 3010” and “5417543010”
• 4012#8888#8888#1881 matches “4012 8888 8888 1881”, “4012-8888-8888-1881” and “4012888888881881”

* Renames WordList modifier to SearchList. Example:  {SearchList=FileName}

* Changes SearchList modifier:

When you save your search list file with the .txt extension in the “SearchLists” subfolder of your current project, you can leave out the path name and file extension from the SearchList modifier. For example, if your project is located in “C:\Users\User\Documents\Aid4Mail\Projects\Project-001\” and your search list file is saved in that location as “…\Project-001\SearchLists\Keywords.txt”, you can invoke Keywords.txt in your query like this:

{SearchList=Keywords}

If your search list file is saved in a different location, you will need to specify the full file name:

{SearchList=D:\Case-123\Keywords.txt}

Adds new search term prefixes:

“C=” to create a case sensitive search and “I=” to ignore character case. The following example will match “U.S.A.” but not “u.s.a.”:

C=U.S.A.

“T=” is similar to “W=”, but produces tokenized searches that are insensitive to diacritics, ligatures, typographic apostrophes, dots and hyphens. Example matching “U.S.A.”, “u.s.a.”, “USA” and “usa”:

T=U.S.A.

The default SearchList type is now “mixed”. You also have a choice of options you can set on the first line of your file to define the properties of your SearchList. This line must begin with an “!” character. The following options and values are available:

• CASE=IGNORE or SENSITIVE
• LITERAL=FALSE or TRUE
• OPERATOR=AND or MIXED or OR
• STEMMING={FileName}

These options will set or override corresponding settings in Aid4Mail. Example:

!CASE=AUTO STEMMING="C:\My Documents\Stemming\English.txt" OPERATOR=OR

In all list types, regular search terms (without the modifier prefix) are automatically tokenized. You can tell Aid4Mail not to tokenize search terms by inserting the following line at the start of your list:

!LITERAL=True

Tokenization applies to punctuated terms, diacritical marks, ligatures and typographic apostrophes, quotes and double quotes. When using tokenization, you must spell words with the correct diacritical marks and full punctuation.

See the “Search lists” topic in Help for full details.

* Fixes a regression causing Python initialization failure.

* Fixes a regression causing the {SearchList[Mixed]=Filename} modifier to fail.

* Fixes several stability and performance issues in the GUI.

* Updates the User Guide.

Version 5.0.10 RC2 (released: July 13, 2021)

* Adds option to process hidden Outlook folders (PST, OST and Outlook profiles). This feature is only available in Aid4Mail Investigator and Enterprise.

* Improves loading of folder tree from Outlook profiles.

* Updates the User Guide.

Version 5.0.9 RC1 (released: June 29, 2021)

* Adds support for Outlook OST and OLM files as source formats.

* Adds support for Apple Mail EMLX files as a source format.

* Adds option to process PST and MSG files (source and target) without needing Outlook.

* Adds option to export all emails to a single PDF.

* Adds many PDF configuration options, including PDF/A file standard.

* Adds fully configurable PDF header and footer using template tokens (e.g. Bates stamp).

* Improves handling of certain types of Thunderbird folders as well as reported progress statistics.

* Improves progress log conversion status information.

* Fixes Received date issue when uploading emails to Office 365 via the IMAP protocol.

* Fixes several issues related to Project management in the GUI.

* Many minor fixes and improvements.

* Updates the User Guide.

Version 5.0.8 Early Access (released: April 27, 2021)

* Adds “Remove journaling envelope” option under Source section in the session settings. Use this feature to process the original message contained in a journaled email.

* Improves reliability of email conversion to PDF files.

* Improves Progress screen by adding the count for unprocessed items and folders.

* Improves the substitution of illegal characters with better alternatives when creating folder and file names.

* Improves log file information. On opening a folder, it includes the total items count. On closing a folder, it shows the count for skipped and unprocessed items.

* Fixes access issues to folders and files located on a network drive.

* Fixes issues occurring sometimes when accessing IMAP folder names that contain one of more of the following characters: * % { }

* Fixes encoding issue when converting emails with UNIX line breaks to Outlook.

* Fixes “Use email UID value” feature (available through the Target “File name” setting). In some cases, it created negative values when all values should be positive.

* Many minor fixes and improvements.

* Updates the User Guide.

Version 5.0.7 Early Access (released: March 19, 2021)

* Adds option in the GUI to export a folder structure to a text file.

* Adds a new configuration field for archival and database formats. Use it to extract just the file name (without the path) of your source:

Source.FileName

or

X-Source-FileName

* Fixes an issue reading source Outlook profiles and PST files. If a system folder like Inbox or Deleted Items contained subfolders, earlier versions of Aid4Mail 5 failed to process them.

* Fixes an issue with incremental processing of Exchange accounts through MAPI and the Microsoft Graph API.

* Many minor improvements and bug fixes in the GUI.

Version 5.0.6 Early Access (released: February 22, 2021)

* Adds Maildir format as a source option.

* Improves rendering the folder structure of Thunderbird mailboxes.

* Reduces loss of emails when processing corrupted source Thunderbird and mbox files.

* Fixes issue accessing Thunderbird mailboxes while Thunderbird is running.

* Fixes issue processing IMAP accounts in Thunderbird when “Message Store Type for new accounts” is set to maildir. If you have an IMAP account stored in the maildir format, you should choose the EML source format in Aid4Mail to process those emails.

* Fixes issue with some MIME encodings of extended characters when using 64-bit Console.

* Fixes issue in filter script syntax when using the “wordlist” option.

* Several fixes and improvements to the GUI.

Version 5.0.5 Early Access (released: January 26, 2021)

* Adds Google Vault (zipped MBOX files) and Proofpoint (ZIP archives) as source formats. Aid4Mail processes these file formats without the need to unzip contents. These new formats are only available in Aid4Mail Enterprise.

* Adds Bates configuration options: number of digits, prefix and suffix, and counter scope (Session, Project, Global). These options are only available in Aid4Mail Investigator and Enterprise.

* Adds option to run license in a VM or virtual environment like Citrix XenApp. Please contact us for details and pricing.

* Adds “Recent projects” menu and improves “Open project” command.

* Adds “Content configuration” and “Email header configuration” editor (available through Edit button above corresponding field).

* Adds “Case” fields to Aid4Mail Converter edition.

* Improves Project Case fields. Values from most of these fields are now available for use in custom header configurations and free-form templates.

* Adds following Template tokens:

{BatesStamp}
{CaseDate}
{CaseLocation}

* Renames following Template tokens:

{BatesCounter} -> {BatesNumber}
{CaseOwnerName} -> {CaseName}

* Adds -cli switch to the Aid4Mail Enterprise Console command-line. Use it when you want to run the console without any session configuration file (.ini). Example (long line might wrap):

a4m-32.exe -cli -App.SessionName:Test -Source.Format:EML -Source.RootLocation:C:\Email\Source\EML\ -Target.Format:MapiMsg -Target.RootLocation:C:\Email\Target\MSG\ -Modify.TargetFileNameFormat:FNV1a -Modify.TargetFolderPathFormat:Flat

{Traffic} -> {Origin}

Incoming -> Inbound
Outgoing -> Outbound
Undefined -> Unknown

{Counter} -> {BatesCounter}
{Date} -> {RawMessageDate}
{DateLocal} -> {LocalMessageDate}
{DateNow} -> {CurrentDate}
{DateSessionRun} -> {SessionRunDate}
{DateUtc} -> {UtcMessageDate}
{Folder} -> {FolderName}
{FolderTop} -> {TopFolderName}
{HashFnv1a} -> {FNV1a}
{HashMD5} -> {MD5}
{HashSha1} -> {SHA1}
{HashSha256} -> {SHA256}
{HashSha512} -> {SHA512}
{SenderEmail} -> {SenderAddress}
{StatusMain} -> {PrimaryStatus}

Type<>Inbox
NOT Type:Inbox
NOT Type=Inbox
-Type:Inbox
-Type=Inbox