Up
What's New in TextPipe
TextPipe makes working with text files easy - fixing websites, converting data
files, extracting from databases, fixing formatting, or
1001 other tasks.
Mainframe file can now trim leading zeroes from negative numbers.
Minor bug fixes and security patches.
Filter panel is now dockable (left/right) or floating.
Trial Input now supports multiple display modes:
- Automatic
- Non-utf-8 data can be displayed (as UTF-8)
- Columnar data can be displayed in a grid
- PDF, Excel and Word data can be displayed as extracted text
- Mainframe data be displayed in ASCII
Fixed validation error on perl patterns ending in $ instead of $$.
Fixed issue with mainframe filter handling of overlapped comp numeric fields
with single digit fields.
Mainframe Master copybook filters now derive the output filename from the
input filename, record type and output type, allowing multiple input files to
easily be processed at the same time.
Fixed issue where changing the file extension in the Output filter corrupted
utf-8 filenames.
Fixed issue where filename browse dialogs such as the Output Filter single
filenames added double quotes unnecessarily.
Fixed periodic reprompting for license key information.
Updated Title Case filter to recognise '1st' and other words starting with
digits as already capitalized.
Updated OEM to ANSI and ANSI to OEM filters to handle newlines correctly.
Fixed Redaction filters for X-over digits except last 4
Added Redaction filters to replace or mask out sensitive field data such as
customer information, medical records, and employee IDs. Safely distribute
reports to partners and customers without compromising customer and employee
privacy, even in heavily regulated industries such as healthcare and
financial services. Redaction filters include:
X-over text
X-over digits
X-over digits except last 4
X-over non-blanks
Replace with blanks
Redact with pseudo NHS
Redact with pseudo SSN
Redact with pseudo bank number
Mainframe filter now diagnoses erroneous characters found in packed fields.
EBCDIC diagnostic dumps are provided with row, column and field name, along
with a recommend PIC 9/X/BINARY clause to fix the issue.
Fixed wrong search/replace filter type being created by COM automation.
Fixed Remove Multiple Whitespace filter under-reporting tab removals.
Fixed error when help invoked.
Fixed Fixed Width to Delimited Wizard not generating correct whitespace
trimming filters.
Resolved license manager connection issues by reverting SSL changes for
license manager.
Fixed memory corruption issue effecting x64 keyed editions.
Fixed memory issue preventing TextPipe x64 from terminating correctly.
Fixed issue with HTML help component.
Fixed file missing from install borlndmm.dll.
New Mainframe Dump filter shows file offsets, EBCDIC hex characters and ASCII
equivalents - making it far easier to debug mainframe file conversions when
the cobol copybook is out of date.
Mainframe filters now handle PICTURE in addition to PIC.
Mainframe filters now handle OCCURS followed by INDEXED BY clauses.
Mainframe filters can now specify a custom EBCDIC to ASCII mapping from the
command line (useful for European languages such as Italian).
Double quotes surrounding pasted filenames now get removed.
UTF-8 BOM is now stripped from the Trial Run area prior to processing.
Fixed multiple issues with file locking for TextPipe Server when multiple
instances of TextPipe are running. Using /MINIMIZED command line option
prevents user preference saving from locking.
Updated "Convert HTML/XML Entities to Text" to fully support HTML5, with
~2200 entities now supported.
File history can now store up to 20 files.
Cancel button during a job now stops the job successfully.
New filter to Remove Diacritics from Unicode text.
Script filter now provides internal 'TextPipe' status object for x64
Script filter now uses syntax highlighting.
Fixed /Q not working.
Mainframe child filters can now use escaped codes e.g. \x00 for record type
indicators.
Fixed proxy port not being available.
Fixed JSON files being saved in ANSI mode.
Fixed error on exit.
Fixed error after a database connection filter completes a dataset.
Fixed bug with command line option /COPYBOOK=
New JSON-based file format which is easy to change, version control and
programmatically generate.
Massive performance improvement for multi-record mainframe copybooks.
TextPipe's filter file format is now saved in multi-line JSON format, making
it human-readable/editable, and super-easy to version control with git/SVN,
and easily see changes. Filter files can now easily be constructed by hand or
programmatically.
Multi-record mainframe copybook layouts are now ultra-easy to convert -
Minimal dependencies between the master and child record structures.
Mainframe BINARY fields are now not quoted (previously they were treated as
alpha fields).
Mainframe copybooks can now be specified completely from the command line -
from the copybook itself (through a reference file), to the header, field
type options etc.
Copybook REDEFINEs now attempt to match the field name followed by (1) if the
original redefined field name is not found.
Mainframe filter now outputs empty fields for any OCCURS DEPENDING ON
iterations that were not used - ensuring that the output file alignment is
retained.
Enhanced the Customize Shell Extensions dialog to allow easier configuration.
Upgraded Unicode support to Unicode 12.1.
Conversion from CSV to Tab now eliminates unnecessary quotes.
Fix for recent files not showing any items.
Fixed Merge Filter not working.
Fixed bug with Extract Delimited Fields filter.
Log now includes filter icons for easier identification.
New filter to Convert Word documents to RTF.
HTML entities are now case-sensitive.
EasyPattern now throws exception for '0 chars' or repeat counts of zero.
The Single File Output option of the Output filter now re-evaluates any
macros in the filename when required.
Mainframe copybook filter now understands 'REPLACES' in addition to
'REDEFINES'.
Mainframe copybook filter now identifies COMP-3 before PIC clause.
Fixed error on Test of mainframe filters.
Fixed display of Unicode data in logging results.
Major UI facelift including new Ribbons and Context Ribbons
Fixed handling of Convert HTML Entities to Text, and updated HTML 4 DTDs to
define 252 named entities.
Split filter now processes macros after the file numbering has taken place.
Fixed 'Filter Options\Include Comment Filters in Log' causing all files to
appear changed, and preventing the 'Only output changed files' setting from
working correctly.
Mainframe copybook filter now understands SYNC keyword.
Mainframe copybook filter can now ignore columns beyond 72.
Fixed issue with Search/Replace list in non-simultaneous search mode where
replacements were to blank.
Fixed @inputFile@ truncating last character of filename.
Updated main memo control to fix intermittent stack overflow issue.
Workaround for Trial Output area not always showing result data.
Fix for Prompt on Replace not inserting correct output text.
Mainframe copybook now supports JUST RIGHT clause.
Mainframe copybook now supplies better error message when COMP- types appear
without a PIC clause or length.
Windows Explorer shell extensions can now be edited.
Enhanced perl regex filter to allow Unicode characters to delimit Whole
Words.
Changed [Delete] action on filter to move to the next filter instead of
moving to the previous filter.
Fixed an issue where backslashes and special characters were not output
correctly if a search/replace prompt was shown and the replace text was not
altered.
Processing tab header now includes a count of warnings if there are no
errors.
Fixed filename or foldername generation when based on a macro such as
@InputFilename@ where the value contained international characters.
Fixed perl pattern ribbon not being available for some filters.
Fixed Remove Columns filter when the end range was 0 (for the last column).
Fixed Trial Output area now showing cursor position.
Fixed Unicode Normalization filters.
Added Restrict to zip internal filename NOT matching pattern list, as partner
for Restrict to zip internal filename matching pattern list, and added help
for both.
Added Search/Replace actions to the Filter Export for better traceability.
Improved Line Numbering filter to correctly tell the difference between start
of file and start of restriction.
Improved error handling for specification filters that do not support
multiple ranges.
Made existing OpenOffice support for ODT, ODS, ODP more obvious.
Fixed @FILENAME@ and @INPUTFILE@ not being recognised.
Fixed context menu macro items not being added.
Updated Prompt On Replace window Info tab to show Unicode text properly.
Updated Captured Strings to substitute Unicode text properly.
Fixed @ENV_....@ macros and @datetime@ etc not being recognised.
Fixed Context menus and Ribbon for Add Header/Footer/Margin filters.
Field specification filters now support field names with embedded hyphens (-
), and field names with spaces can be used by surrounding them with quotes.
A warning is now output when a field specification does not match the field
names found in a file.
Improved error handling for script filters and loading settings and
Languages.
TextPipe Engine now does not show popups.
Updated Scripting Filter to TextPipe x64.
Restored Scripting Filter to TextPipe x64.
Added help links for IoT Hub/Event Hub filters.
Text to Word List now recognises English possessives (or other abbreviations)
ending with 's.
Updated Unicode compose/decompose functions for NFC, NFD, NFKC, NFKD.
Updated PDF libraries.
Updated code page converted library (2013->2017).
Fixed UTF-8 case-sensitive sort.
Fixed Convert ANSI to UTF-8.
New Azure Event Hub connection for JSON data.
New Azure IoT Hub connection for JSON data.
Fixed CSV/Tab file processing where the number of fields in each row was less
than those specified in the header.
Updated Remove Blanks from Start of Line/End of Line to handle UTF-8 e3 80 80
IDEOGRAPHIC SPACE, common in Chinese text.
Fixed bug in CSV parsing.
Fixed bug in delimited-field restrictions that were not moving or deleting.
Fixed bug in delimited-field restrictions using a range with an end-field
relative number.
Column specifications for Delimited field delete, extract and restrict can
now specify multiple columns in one filter e.g. 6, 9, 61..63. Field names can
be specified (if 'first row contains field names' is checked)
Excel column numbering - prefix Excel column name with 'COL_' e.g. COL_A ..
COL_AB.
Updated mainframe copybook filter to allow COMP type before the PIC
The Break on Value Change window now has a checkbox 'Break on Value Change'
that can be unchecked to control the actual filter during a job.
Changed Filter Log, "Don't log to file if line count is less than" field to "
Log errors to file after X errors"
In non-interactive mode (cli), the existing logs are now transferred to the
log file when the visual log hits the limit.
Fixed position of prompt on replace window, so it always pops up in a
consistent location.
Help file updates.
Added new Run section to Ribbon.
Copybook filter and Database filter now change output extension to match
format.
Added JSON and SQL INSERT script output format to copybook filter.
Added JSON output format to database filter.
Added Convert Tab to JSON filter.
New command line option and Windows shell extension to 'Prepare filter for
distribution' on a set of files.
Added /openOutputOnCompletion command line option, with set, unset or toggle.
Added /CLOSE command line option - useful for automated processing.
Updated /SAVEFILTER to overwrite current filename if no filename specified.
Streamlined right-click menu in Filter Library.
Previous filter name in Windows title is now cleared when a filter is closed.
Default to UTF8 encoding instead of ANSI when loading/saving files.
Updated case changing filters to work with UTF-8 encoded text.
'Only log when a search/replace occurs' checkbox now properly actioned.
Changed Convert Word/Excel/PDF to text filters to output UTF-8 text.
Extract URL filter now copes with ANY scheme, from the previously supported
mailto:, http:, https:, nntp:, gopher:, ftp:, ftps:, and new ones such as
call: and skype:
Fixed Mainframe copybook filter handling of single line REDEFINES.
Fixed Add Left/Righ - Margin, Header/Footer filters not saving defaults.
Fix for CSV/Tab field names not being extracted from files.
New warning on blank PDF conversion output.
Fixed job termination 'Parent window is nil' message - occurred when the log
filename path was not fully specified.
Fixed Line Number filter 'Reset at end of file' not working, and being the
reverse sense.
Fixed Search/Similar text options not being available.
Added 'ignore leading characters' to Mainframe Copybook filter to help
resolve common copybook processing problems.
Added ability to save and load Shell Extension filters to tab/csv.
Upgraded PCRE regex library.
Command line wizard - fixed copy buttons 2-5 not working.
Enhanced capability of search/replace list filter's edit grid.
Enhanced log output to provide information for every filter type - very
useful for filter debugging.
Fixed issue with "Restrict to internal zip filenames matching" filter that
prevented text passing through the filter in trial run mode.
Fixed issue with exit code being reported that was outside the specified
values.
Updated help file for new Ribbon interface.
Improved speed of Mainframe Copybook Filter.
Improved speed of Exact match search/replace (internally reverted to older
non regex code).
Updated Mainframe Copybook filter to handle BINARY fields in
B. notation with the correct length.
Remove BOM filter now detects if it has changed the file or not, and handles
UTF-16 LE properly.
Fixed Move Delimited Fields filter not working for single columns.
Fixed job not cancelling when requested.
Fixed Unicode checkbox obscuring other controls.
Fixed combo boxes being shown with a dimmed background color.
Fixed handling of field selection when the first field is relative to the
number of fields (ie a -ve offset). Fixed handling when a relative reference
is used with a field count.
Fixed field selection when erroneous ... used for a range.
Fixed bug when Enter key pressed in File grid.
Fixed tree view hot keys being global.
Fixed search/replace list filter UI.
Mainframe copybooks now allow + and - VALUES clauses.
Installer for x64 systems now also installs x32 shell context dialogs
Fixed greediness setting not being respected for perl patterns.
Fixed delete behaviour for multi-level filters.
Added Ctrl+Up and Ctrl+Down to move filters up and down in the filter list.
Added Shift+UP and Shift+Down to move filters up and down in the filter list
but at the same level.
The Filename is now passed to subfilters of a Restrict to Matching Lines
filter.
Copy/Move CSV fields now works again.
Fixed resources being exhausted with multiple trial runs.
Added Ctrl+D to duplicate the current filter.
Added Ctrl+E to Escape the text in the current edit field e.g. to prevent \
or @ characters being treated as special.
Added short cut list to help file.
Fixed bug with handling of negative offsets (relative to end) for field, line
and column restrictions.
Improved resize behaviour.
Restored Ctrl+O short cut on File\Open.
Fixed utf-8 mode not respected on Search/Replace List filters.
Removed Buy Ribbon from registered version.
Fixed Map filter display and click-to-edit not working.
Added 'Minimize Ribbon' option to preferences.
Fixed Capture Variable, Yes To All button not working.
Fixed field extractor not correctly handling two consecutive named fields,
and leaving off the last field.
Recent files list now shows filename first, folder second, as workaround
until list width can be adjusted.
New modern Ribbon interface. Major advantage being easy discovery of the huge
array of useful helpers buried in the right-click context menus are now
available to other than power-users!
Select, Copy, Move and Delete Delimited field filters now use a powerful new
field editor to determine which fields to operate on.
Use field number, field ranges like 4 .. 10, Excel column names like COL_A or
COL_J .. COL_Z, as well as field names from the file.
Field names do not have to be in order - TextPipe will figure out where they
are in each file and automatically use them.
New filter to Extract Delimited fields - simply list the fields you need, and
TextPipe will extract them.
Extract field names from files directly in the file grid - just highlight the
files, right-click and choose 'Extract field names to clipboard'. TextPipe
puts a sorted field list on the clipboard.
Match entire lines/records of CSV, Tab or Pipe data and output to a new
format simply. Match an unlimited number of fields, use custom delimiters
too. Perfect for mainframe data with thousands of columns.
Filters Menu \ Replace \ Match CSV/Tab/Pipe record.
Easily generate patterns for CSV, Tab or Pipe-delimited entire record
matches. When using an EasyPattern match, right-click the Find What field and
choose 'Insert EasyPattern \ Line Patterns \ CSV record'. Also Tab record and
Pipe record. Matches up to 50 fields.
Mainframe copybook filter now gives exact line and columns position for
errors, and displays the current line/column when editing.
Mainframe copybook filter now handles single-field REDEFINEs better.
Mainframe copybook filter now allows ':' as part of fieldname.
Fixed ASCII warning detection logic with zero byte files.
Easily compare results in Excel, by taking advantage of our Excel-style
column naming in replacements.
So now instead of having to use $1$, $27$, $26$ to identify matched fields,
you can now use Excel-style column names like @col_A@, @col_AB@, @col_Z@.
These new names are now available for every search/replace throughout
TextPipe. You can also insert excel-style column names easily using the
right-click menu on pattern replacements - right click, Insert match
subexpression \ Excel column numbering \ @col_A@
New Sort by UTF-8 (case sensitive and insensitive).
Filters can now be paused by clicking the [Go > ] button e.g. to receive a
skype call.
Increased filter history from 10 to 36 entries.
New warning if the filter being opened is already open in another window.
New button that allows defaults for each filter type to be set under user-
control rather than auto-magically. This prevents hard-to-debug issues with
pattern match options that are normally hidden.
Moved filter text search to new location at top of filter tree, normally
hidden.
Renamed 'Repeat Subfilters X Times' to 'Run Subfilters X Times'.
Fixed Changing 'Include comment filters in log' not flagging the filter as
changed.
Now checks for 'zombie' filters that follow a T-filters' secondary output
filter - these filters do nothing.
New install option to pin to taskbar.
Filter log filename can now include macros like @fullInputFilename@ and
@filterName@, allowing log files to be grouped and named with their filters
like
@filterPath@@filterName@.log or
%HOMEPATH%\log\@filterName@.log
Changed macro @FILTERFILENAME@ to exclude the path, to be consistent with the
naming of other macros and to generally be more useful.
Search/replace list filters now support Unicode.
New option for Add Line Numbers - to reset at the start of a new file.
Removed unused search/replace types and re-prioritized list of types.
List of filenames can now be Unicode.
Split file filters will now remove the last file if it has zero bytes.
EasyPatterns are now UTF_8 enabled by default. Now have UTF-8 mode setting
for binary/ebcdic data.
Updated ZipMaster component to 1.9.2.
Updated PCRE regex to 8.6.9.
Fixed bug with third option of line range selection - referencing lines
relative to the end of file with a given number of lines.
Downloading URLs in the file list now supports https, as well as redirection
from http to https and vice versa.
Speed improvements.
Fixed loading/display/handling of Map filters.
Moved import/export of search/replace lists into the search/replace list
filter.
Improved handling of tab delimited data with embedded tabs (in quotes), for
Convert Tab-delimited to CSV, and Restrict to Delimited fields (Tab) filters.
Filter options, log file view and delete now correctly change the current
directory to the same directory as the filter before proceeding.
File Menu option to rename a filter.
Fixed example filters not being copied after an uninstall.
Fixed help file links for Convert Tab/CSV (with headers) filters.
Cleaned up interface and installer.
Processing speed improvements.
Fixed error on loading Map filters.
Re-arranged UI for better user experience.
Loading a file into the Trial Input area now does not double backslashes or
other special chars.
Temp files used in processing Zip files, or in Convert Word/Excel/PDF to Text
are now thread-safe to avoid conflicts.
New error on zip DLL missing.
Download of web pages in the file list now occurs with user agent of
'Mozilla/5.0' - previously this used 'Internet Explorer 5.5'
New macros to allow finer granularity to dates in filenames etc: @_day,
@_month, @_year, @_hour, @_minute, @_second.
Fixed multiple TextPipe Engine (COM) issue on running and then closing
filters.
Now includes COM import manifest for TextPipe and TextPipe Engine to allow
for registry-free COM.
Added new COM function appendFilter().
Split filters can now cope with Unicode filenames.
New Tools Menu item to search your filters for a piece of text. Perfect for
finding filters in a large filter list.
New Replace options - replace (or skip) rest of folder. Great for websites
and other hierarchical structures.
New command line option /EXPORT= to output a UTF-8 text
representation of the filter. From here it can be compared using a diff
utility or stored under source code control. Very useful for comparing two
similar filters to see what exactly has changed, e.g. to generate exports for
all filters in the current folder:
FOR %f IN (c:\folder\*.fll) DO "c:\Program Files
(x86)\DataMystic\TextPipe\TextPipe.exe" /HIDDEN "/f=%f" "/EXPORT=%f.txt"
(In a .BAT file, remember to use %%f instead of %f).
Fixed export of Search/Replace List and Search/Replace filter.
/EXPORTFILTERTOLOG now uses UTF-8 format.
Fixed Restrict to Filenames Matching/Not Matching Filters, bug introduced in
v9.8.
New EasyPattern [MacAddress] for firewall log processing.
Massively improved speed of filters that match against a list of patterns e.g
extracting lines that match a list of 20,000 words.
Mainframe filter now detects and warns when ASCII files are processed.
New Expand All/Collapse All in right-click menu of Filter view.
If the filter list is running and F9 is pressed, TextPipe now changes to the
Status tab.
Fixed: Right-click, Escape Special Characters, did not set the filter to
modified.
Comment filters now have horizontal scroll bar.
Clearing the filter search using [X] button now selects search field.
Fixed File\Export filter to clipboard, output filter was missing line feeds.
Inappropriate menu items are now dimmed when a non-filter window is active.
Filter is now no - marked as changed when expanding/contracting.
Open file on completion now uses the default editor if no file association is
found.
Better bounds on Fibonacci buffer growth.
Fixed opening of web page after File Menu\Export to web page.
Removed warning about active COM objects.
Fixed bug in Remove lines matching perl pattern.
Difference tool integration - new function to open a file in text difference
tool - in Trial Run, from the Files Tab, and from the Status Results Window.
Integrate with tools such as WinDiff and BeyondCompare.
Sort function now much faster.
Removed locking of TextPipe into the COM Running Object table, so COM errors
do not occur on completion of scripts.
Upgraded PDF component.
Display text now outputs 'simultaneous search' and 'Process longest strings
first' options.
Fixed bug in Convert Excel to Text filter - was ignoring non-hidden sheets.
Mainframe copybook filter now allows pass-thru EBCDIC data ie BINARY data
e.g. using PIC X(48) USAGE IS BINARY.
Mainframe copybook filter now does not insert a null (\x00) character after
fields in Fixed Width mode.
Invalid copybooks now send errors to log when running from command line.
New EasyPatterns:
[Drive] e.g. C: E: -> returns drive letter in @drive@
[Folder] e.g. \My Documents\ -> returns 'My Documents' in @folder@
[Path] e.g. d:\My Documents\ -> returns 'd:\My Documents' in
@drive@ and @path@
[UNCPath] e.g. \\server\share\path\filename -> returns @server@,
@share@ and @path
[Filename] e.g. testdocument.doc -> returns @filename@
Unicode conversion filter did not retain the Error Character if the Input or
Output encodings were not changed.
Hidden worksheets are now ignored by the Excel to Text filter.
Split filter now cleans up its temporary files.
Trial run area now shows output progressively - giving important feedback
earlier in the filter development lifecycle.
Now shows '(more pending)' next to the file count if more files remain to be
counted, so users aren't misled that only 1000 files will be processed.
Added new filter list menu items for Remove exactly, Remove perl pattern, and
Remove EasyPattern
File grid now allows mass-changing of Action or Subfolder columns using
right-click menu after selecting rows in the row number column.
File grid ensures that the top blank row is used with drag/drop.
Pressing [Tab] in a memo control now inserts a tab - it no longer changes
focus to the next field.
Updated Perl regexp third-party package.
Updated PDF third-party package.
Re-ordered scratch pad and gave tabs consecutive numbers.
'Delete' action of search/replace grid now much faster for entire-grid
clearing.
TextPipe now warns if a Convert Word/Excel to Text filter is used at the same
time as the Input Filter/Process Inside Compressed files option - these are
not compatible.
Fixed missing comma in VBScript/Jscript code generation for search/replace
lists.
Installer - fixed x32 shell extensions missing under Windows x32.
Replace List path is now relative to the filter path.
Using File Menu\Open to open non-filter files, now adds them to the current
Files To Process list.
Fixed error with Restrict to Blocks of Length X and several other filters
causing out of memory errors, as they were confused for 'repeat filter X
times' filters.
Fixed 'Only log when a replace occurs' not working for consecutive search
replace filters in the Search/Replace list filter.
Fixed UTF-8 mode perl pattern replace strings from being converted to utf-8.
Added Help entry for Repeat Subfilters X Times.
Adding filters with no valid filter window now creates a new filter window.
When Utf8 Support is checked for replace filters, Unicode characters in the
search for and replace with fields are now correctly converted to UTF-8 prior
to being processed by the PCRE engine.
Clicking [Delete] on empty areas of the Files grid no longer causes errors.
Added log entries for size/date of ignored files.
Search/replace lists can now generate log entries - useful for debugging.
Also, logs can be output only for where replacements occurred, which keeps
the log size manageable when importing large search/replace lists (default On
when loading existing filters as previously no logging was done).
EasyPatterns v2.8 - new keywords EBCDICletter, EBCDICupper, EBCDIClower,
DayNameShort, DayNameLong, MonthNameShort, MonthNameLong, YearShort,
YearLong, Float, Integer. WordBreak is a new synonym for WordBoundary. The
behaviour of number, numbers and numeric has been changed so that instead of
matching a single digit (which is already handled by the digit keyword), they
now match a number with an optional sign and optiona - decimal point.
Mainframe copybook filter now copes with VALUES of ZERO/ZEROS/ZEROES,
SPACE/SPACES and multiple comma-separated values in addition to the string
and integer values previously supported. Now supports VALUE clause on numeric
fields.
New right-click menu items in all filters view, to Link To or Insert filter
files.
Filter comments can now be included in the output log, from the filter
options settings at the top of the tree. This greatly assists in debugging
complex filters.
New filter to Repeat Subfilters X Times; useful when you are not sure how
many times a search/replace pattern needs to be applied.
Named subexpressions created by a Split on Pattern filter are now saved as
global variables for use in other filters. Useful for pushing split pattern
text into filenames.
If Prompt On Replace is disabled during a filter list, the filter is now
marked as modified when the Prompt form is closed.
Search/replace lists now discard blank search terms and terms where the
replacement is identical to the search.
Log filename now has environment variables resolved before display.
Partial trial runs now reset the status window properly.
Better handling of split filenames with format string errors in them.
Filter components relative to the filter path now work in Unicode folder
names.
Restrict to Inside filenames is now available in the Lite version to assist
batch file re-naming.
Upgraded PDF component from 8.15 to 9.13 - now handles a greater range of PDF
document types.
Updated to Zip Master 1.9.1 and PCRE v7.5.
Fixed handling of EasyPattern Ascii() keyword.
Fixed bug with Right-click\Delete on files in the file list.
Fixed bug in search/replace list, in perl pattern match mode. If the search
term contained brackets (), then replacement phrases on rows following it
were mismatched.
Split filter now copes with absolute pathnames in the split filename. Split
filter now outputs log entries when it starts a new file, and outputs
separate logging details for each split file.
Number of files processed now does not include files inside zip containers.
The current file being processed now shows the zip container and the internal
filename.
Fixed export of Search/Replace lists filters to VBScript.
Fixed incorrect path for shell extensions on install.
Fixed bug with handling of moving a delimited field, where the line did not
end with the expected delimiter.
Bug fix for Retain modified dates for the compressed file.
Matrix search now uses slightly faster initialization method.
Fixed installer issue on x32 systems.
Retain modified dates now works for compressed files (for the compressed file
itself).
Matrix (parallel) search/replace for Exact replacements:
- now supports First Only option - any term that has already been replaced
will not be replaced again.
- now supports Case Sensitive replace- where the case of the replacement is
based on the case of the found text.
- now supports prompt on replace.
- now much faster - 600,000 replacements against 8MB compressed .xlsx file in
31 seconds.
- fixed bug where changes were not counted and hence the output file was
never updated unless changes had been made by other filters.
- fixed bug where text that did not match was output in lowercase when Match
Case was unchecked.
- fixed bug where text at the end of the file or restriction was not
replaced.
Added .XPS as a compressed document type.
Word to Text Filter and PDF to Text Filter now set output filename to .txt,
Excel to CSV filter now sets output filename to .csv.
New warnings to ensure Process Binary files is checked when Process Inside
Compressed Files is checked.
TextPipe now switches immediately to the Status tab when a trial run is
started so that progress can be reviewed, and then switches back to the Trial
Run tab on completion.
Exact search replace now does not even allow special characters.
New status messages from search/replace list filters to give progress updates
when loading large files.
New filter to restrict processing to specified files within a zip or other
compressed file.
Compressed file handler now removes any left over files at the end of the
filter job.
Compressed file handler now shows status information as it extracts and re-
compresses files.
New status logs for filters that perform end-of-line conversions.
New validation check for 'Send variable N to subfilter' to ensure there are
at least N captured expressions.
Now handles any PKZIP compressed file types regardless of whether the
extension is known or not.
Fixed bug with handling compressed files where modified files were not
updated back into the original compressed file.
Fixed bug where AV occurred when adding a new file to a file list that had
been cleared using [Clear All].
Fixed export to clipboard of search/replace lists - the second and subsequent
lines were not indented correctly.
Removed 'Warning: Find and Replace string lengths don't match' message when
the Action is not set to 'Replace'.
Fixed problem with endless loop caused when Up or Down arrow pressed in Files
Grid.
File grid now supports Unicode characters.
Support for Unicode pattern matches in the files grid (perl regex mode).
Split filter now allows Unicode filenames, and Unicode file break patterns.
Grep filters now allow Unicode patterns (when UTF-8 support mode is enabled).
Fixed installer error with registry entries required by x64 Shell Extension
Fixed crash with sorting more than one file.
Fixed crash experienced by some users caused by the x64 shell extension when
the desktop was right-clicked.
Massive performance improvement for large added strings, such as headers and
footers.
Mainframe copybook filter now handles the option 'Process numeric fields
only' properly - previously it output blanks for alpha fields in this mode.
New popup menu for the status log - options to copy the log entry, copy all
log entries, open the referenced file or its containing folder.
New filter Convert HTML/XML Entities to Text, as it handles both numeric and
253 named entities.
New 'Copy to Clipboard' button for Search/Replace list filter.
Right-click menus now display available environment variables and their
current values, for easy insertion.
Updated internal PCRE (Pattern Matching) engine to v8.30 and support for
Unicode 6.1.0.
The perl regex position '^' for the start of the file now works with Unix
files.
TextPipe Engine now returns an error if the output file is open in another
application - in the same way the GUI does.
Improved COM object model to have less embedded references to objects, making
it easier to free the objects. Also, calling the output* and input* functions
when calling the TextPipe.FilterWindow object no longer result in errors.
Uninstall from now on will not remove settings.
'updateDate' in registry can now be set to zero to disable update checking.
TextPipe Engine now ignores Ctrl+C and Ctrl+Break.
MIME Base 6 - Decoder now excludes the trailing garbage character.
Output Filter now allows macros and ENV vars in the single output filename.
Fixed threading errors when cancelling running jobs.
Debug window and embedded help windows now record their size correctly.
Stack overflow, as in the case of pattern matches with poor constraints, are
now trapped as Exit Code 254.
Fixed environment variable substitution error in filenames.
Added new macro @inputPath to contain the path to the input file, with a
trailing slash. This lets you easily create output folders at the input path
location.
Added new macro @inputExtension to contain the extension of the current input
file.
Output Filter's Single Output Filename, new File Extension and New Path can
now all contain macros.
Added support for Open Office Writer (.ODT), Calc (.ODS) and Impress (.ODP)
file types as new compressed file types.
HTML Restriction filter now supports entity names with embedded colons (:)
and hyphens (-) which is essential for working with Open Office files and
other XML types.
EasyPattern keyword [backwardSingleQuote] renamed to [graveAccent] as the
original name was incorrect.
The right-click context menu on the Filter Tree now allows groups of selected
filters for 'Make Into a Subfilter' and 'Promote'.
Fixed operation of log 'overwrite' option - the log cache would not clear
previous logs created since TextPipe was started. Logs were also not being
closed at the completion of a filter job, hence they could not be deleted.
Macros and environment variables are now allowed to contain numbers, provided
that the first character is not a number.
The right-click context menu of the replace string now inserts captured
variable names instead of $1, $2 etc when you ask it to insert a CSV, Tab or
XML string. This makes it far easier to identify which fields belong where,
as well as making the replacement string less sensitive to changes to the
search pattern.
New optimization for named captured variables - TextPipe replaces these with
the captured variable number behind the scenes - so you get the ease of use
benefits of named variables, with no spee - decrease.
EasyPatterns now allow a literal (ie a string of characters) inside a 'not'
expression, e.g. not 'ABC', or not EBCDIC('HDR').
Mainframe copybook filter can now remove FILLER fields inside OCCURS blocks.
Supported but not installed code pages now show 'NOT INSTALLED' beside them.
Fixed AV's when saving or loading a file when running under Windows 7.
Sort filter set to maximum allowed memory of 2GB to avoid integer overflows.
New filter to perform a calculation on restricted text e.g. add 1.5 to a
field.
Comment filter now allows Unicode text.
Windows Home Server no longer considered a 'Server' platform.
Log files now support Unicode.
Fixed action of 'Skip/Replace in Rest of File' when used inside a restriction
- previously it carried forward for the rest of the job, instead of the rest
of the current file.
Remove multiple whitespace filter now generates detailed log information.
Trial Runs now generate the same logging data as real runs.
Fixed missing filters from Lite Edition.
TextPipe only registers its COM object during install and for the /REGSERVER
command line option - preventing errors on less-than-admin use.
Restored action of F9 button to start running when the All Filters view is
selected.
Opening and Updating zip file log messages are now clickable.
Updated third-party zip DLL to improve Windows x64 support.
Fixed handling of double backslashes in the replacement term for
search/replace lists when consecutive search was selected.
Unicode strings are now converted to UTF8 when the perl pattern filter has
UTF8 mode selected.
Restrict to matching/non-matching filenames now works with files of unlimited
size.
Fixed Sort filter allowing duplicates.
Entering \x{1234} when UTF-8 mode is enabled for a perl pattern match now
does not show a validation message.
Updated PCRE engine.
Fixed scrolling of found text into view in Prompt On Replace dialog.
New [Add Files] button on Files to Process tab.
Combo boxes now work with right-click context menus.
Search/replace list import now supports Unicode.
Updated PDF libraries for extraction from more recent file formats.
Added large [Add Files] button to Files to Process tab.
Capture Text, Break on Value Change window now shows length of strings and
current cursor position.
Updated internal PCRE (Pattern Matching ) engine to v8.13 and support for
Unicode 6.0.0.
Updated Unicode internal libraries to support Unicode 4.1 for Normalization
etc.
COM callees are now notified of Stack violations (e.g. during pattern
execution) or other critical errors via the existing 'FilterWindow.errorText'
variable.
Only modified files are added back into Zip files such as .zip, .docx, .xlsx
and .pptx.
Updated x64 extension to only show current file name, not full path.
Better checking for changed files with Extract matching/non-matching lines
and Remove matching/non-matching lines.
All Available Filters tree now show properly when the text 'Search...' is
translated.
Fixed errors with TP Engine not unloading cleanly, and not returning data.
Windows Explorer x64 extension working!
Fixed dimming of unavailable filters in Lite and Standard versions.
Fixed Browse buttons not resizing correctly.
Converted file list field to Unicode.
New 'Close' buttons in basic forms (editor, copybook tree viewer).
Now handles 'Convert Word/Excel to text' inside subfilters, prior to the job
starting (e.g. for processing exclusion lists etc).
Fixed Mainframe copybook filter handling of OCCURS .. DEPENDING ON .. when
the Trim Fields option was checked.
Fixed display of Unicode text in various multi-line edit fields.
Fixed crash on filter run when working with files.
New 'Open Containing Folder' for filter lists.
Fixed bug when double-clicking on filenames.
Filter search now works for filters like 'Sort' - where previously the
invisible short cut character & prevented this from working.
Search/replace lists now returned to na-ve UTF-8 handling.
Binary test now works for Unicode filenames.
Fixed bug with the way that PlaySound was called at the end of the filter job
which caused crashes.
Removed windows explorer x64 extension as buggy.
Search/replace lists now check for Unicode external files.
New search function to quickly find matching filters or external files.
Simultaneous search now works with patterns - not just Exact match.
Filter move Up/Down buttons in short cut menu.
Enable/Disable prompting in short cut menu to easily debug large groups of
filters.
Secondary output filters are no longer removed when pasting.
Log files are now opened in their associated app rather than in Notepad when
double-clicked or when [Show] button clicked.
Better Unicode filename support when filenames are launched by double-
clicking on text fields.
Updated internal pattern matching libraries.
TextPipe Engine now logs just as the GUI can.
Unicode filenames now supported by Merge Filename; Output Extension, New Base
Folder and Single Output Filename; Split Filename; Log filename; Map comment;
String filters' filenames, Search/Replace List filename.
Unicode filename support in status grid, file grid, processing etc.
Dimmed filters now cannot be double-clicked to add them.
While loading a search/replace list, TP now responds to events.
Link Filter now in every edition.
List filename now gets loaded properly.
Added .ODT as a compressed file type so that OpenDocument documents can be
handled just like Office .DOCX format compressed XML files.
New Files to Process field to load files from a list file (note - this
ability was already available from the command line with /L= and from COM via
addFileList).
The Replace List filter now has a button to paste a list from the clipboard.
Added new exit code of 255 for abnormal termination due to textpipe.exe being
killed or an unexpected error.
Deferred hooking into Windows HTML help system for faster startup.
Migration of filters from Pro to Standard, and Casing filters from Standard
to Lite.
New file date and file size filters to help select files to be processed.
New context menu items for filter list, to duplicate, demote or promote
filters.
Renamed variables in Extract-generated .vbs and .js scripts for consistency
and readability.
Lite and Standard version now show dimmed menu items for unavailable options.
Upgraded PCRE pattern engine.
Improved speed of double-quoting and trimming mechanism in Mainframe Copybook
filter.
New EasyPatterns [DayNameLocal] and [MonthNameLocal].
Default for loading older Mainframe Copybook filters without Force Quotes
options is now set to unchecked.
Fixed EasyPattern keyword 'atomic'.
Fixed coInitialize/coUninitialize COM errors with repeated running of the
scripting filter.
Comment filters and other filters with multi-line values now display just the
first non-blank line in My Filter List.
Various GUI fixes.
New macro @outputFilename to output the name of the output file without the
path.
Correct detection of Windows XP x64 as a non-server O/S.
Updated code to detect My Documents and AppsData folder under Windows 7.
Convert Excel to Text filter now outputs a form feed (ASC 12) character
between worksheets.
Filter lists are now freed on completion. E.g. Filters that load large
suppression lists (e.g. 300MB Remove matching lines filters) can now be run
repeatedly without restarts.
Better error trapping for Out of Memory errors and other exceptions that
occur during filter execution.
Many memory leaks fixed, making working with large files multiple times far
easier.
Speed improvements for Remove Lines From List, and Retain Lines From List,
when using large external compare files (10MB+).
The Output Filter setting of 'Backup Mode' now does indeed rename the
original input file to '.bak' rather than operating on the output filename.
Setting the global variable @CLIPBOARD using a Capture Filter or a named
variable in a search pattern now sets the Windows clipboard as well.
Missing Recently Used files are now removed from the list.
New 'atomic' keyword allows EasyPatterns to match either all or none of a
group e.g.
[ atomic(0+ spaces) ]
matches all the spaces that are present, or none of them - and does not
backtrack and keep retrying with one less space each time. This makes
matching far faster.
Mainframe Copybook Filter in the Server Edition no longer automatically adds
a Split CPU filter.
Shell extension now supports Unicode filenames - fixed error with only the
first letter of menu options being shown.
Mainframe copybook filter now allows trailing minus sign to identify signed
fields e.g. PIC 9(11)V99-.
Fixed problem where status messages were occasionally lost or doubled up.
Improved Unicode file handling.
Fixed bug with output files causing file rename errors.
Sound on completion when run from command line now uses settings from
Preferences (rather than being forced on). Can be overridden using
/PLAYSOUND.
Updated Explorer Context Menu to support Unicode.
Now uses updated zip library.
Partial support for Unicode filenames.
Fixed bug with Ignore Case always enabled, for Count Duplicate Lines and
Remove Duplicate Lines filters.
Fixed bug where @ and $ signs in the replace string were handled incorrectly-
when used in the Search/Replace list filter in sequential mode.
Change to update mechanism to prevent spurious update notifications.
Log file is now closed and re-opened every 30 seconds so that the file size
increases over time. If you open the file it will still show the currently
written data.
Fixed input file gathering - new files are now available for processing
immediately rather than grouped by folder. Previously, folders with thousands
of files would cause an apparent stop in processing.
Uninstaller now does not require explorer.exe (Explorers and Task bar) to be
closed; reboot or logoff required as before.
Cancel button now does not pause thread until [Ok] clicked, due to conflicts
with suspending the thread.
File list is generated in parallel with processing - so work commences
immediately, even with millions of files to process.
Status window now shows expected completion time based on total data volume
being processed.
Mainframe Copybook Filter - adding quotes to alpha fields is now an option
rather than automatic. Quotes and field delimiters in the record header and
numeric fields are now escaped appropriately.
'Delete Input Files On Completion' Input Filter warning message now disabled
in command line and Engine mode.
Scripting filter now available in the Standard version.
Performance improvement for Switch CPU filter.
Restrict to matching lines filter now uses its subfilters to determine if a
change has occurred to the input file.
Fixed bug with Split CPU filter when used with multiple output files.
Fixed bug in MultiCPU version with Split CPU filter stopping output files
being written.
Merge filename is now added to the COM output files list.
Fixed TextPipe Engine bug with Edit Distance search/replace and Convert to
Word List not working on first filter execution.
Fixed Insert Filter option changing filename and inserting extra Output
filter.
If the Output file cannot be written to (e.g. the network resource is no
longer available), then the Input filter option to Delete Input Files is
ignored. The Open Output Files On Completion option is also ignored in this
case.
When run from the command line, errors (such as no matching files to process)
now do not require manual intervention. They are available in the log if it
is turned on.
If TextPipe cannot create a temporary file on the target drive (e.g. if it is
a network drive), then it falls back to creating it on the local machine in
%TEMP% and copies it across at the end of the job.
Fixed bug with HTML filter when 'Include text' is unchecked, the restriction
is designed to match text between open and close tags, and the open tag has
no text after the tag name e.g. ...
Split filename can now include macros using @varname syntax.
Database filter now copes with multiple record sets from a single query.
Changed installer to not require admin, for Vista and Windows 7.
TextPipe now does not require admin rights to run - which in Windows
Explorer:
- restores ability to double-clicking a filter to open it
- restores the TextPipe context menu
- restores drag and drop.
Clipboard input and output can now both be in Unicode format.
New COM method terminate() to allow filters to be stopped when called from
execute() or executeClipboard().
TextPipe Engine now supports the Split CPU filter.
Fixed bug that occurred with T-Filters and Uppercase/Lowercase filters. The
Upper/Lower filters overwrote memory so that modified text could occur in the
alternate stream of the T-Filter.
Fixed filter 'Restrict to filenames matching pattern'.
Mainframe copybook filter - now displays a red bar at column 72 - any data
beyond here gets truncated in line with mainframe commenting standards.
COM function addSimpleFilter() now exports the isUnicode option.
Much more complex pattern matches now allowed - increased limit of capturing
patterns from 99 to 65535.
Removed environment variable capability from the split filename because of
conflicts with %f, %e and %d macros for the file name, extension and number.
Mainframe copybook filter - alpha fields now have quotes added around them,
and existing quotes are doubled up to escape them.
Environment variables now allowed in the split filename.
Added [alphanumeric] and [numeric] to EasyPatterns.
Fixed validation when a Restrict to Filename NOT matching filter was missing
subfilters.
Fixed bug in invocation from shell extension.
Modified TextPipe and TextPipe Engine so that error messages are now
available through the ErrorText variable when called via COM.
Output file size was missing from the log.
Fixed Unicode Search/Replace error that caused file corruption beyond
64Kbytes.
Fixed error with clearing of output strings.
Script errors that occur at Run Time are now logged as warnings in the log
rather than stopping the filtering.
Fixed error with parsing of output strings.
Fixed shell extension not starting filters correctly.
New 'Prepare Query' option for database filter to make SELECTs more
compatible.
Fixed elapsed time display when duration was less than 1 second.
New Edit Menu\Preference to control if Word Wrap is on by default in edit
controls.
New command line option /fromshellext to indicate if a filter is being run
from the Windows Explorer shell extension. Existing files in a filter
triggered from Windows Explorer are now cleared and a warning given.
Added new COM function addMainframeCopybookFilter() to recreate Mainframe
Copybook Filters.
New installer should allow overwrite of existing installs and/or uninstall
without reboot.
Files to Process, and Search and Replace List filenames can now be specified
relative to the current filter's path.
Improvements to Customize Shell Extensions dialog.
New option for Grep filter to ignore empty matches.
New pre-compiling of replace expressions for faster replacing.
New 'Break on Change' option for captured variables makes debugging far
easier.
Revamped the simultaneous search/replace filter to work with lists of
300,000+ replacements. New Longest First option.
Copybook filter now copes with nested OCCURS where the nesting contains only
one field (special case).
Copybook filter now copes with OCCURS DEPENDING fields.
Copybook filter now outputs the header only once - very useful inside the
restrictions that are required for processing variable format files.
Copybook wizard - fixed disabling of trailing delimiter on last field.
New command line option /CLOSEONCOMPLETION - which closes the filter window
when a thread completes. This is typically used with a /Z command line file
and a large set of filters.
New option for Count Duplicates Filter to change the format of the results -
e.g. put tab between the count and the string, reverse the order etc.
Grep filter now has an option to match UTF-8 characters for Unicode text.
Convert to Word List filter now allows words with numbers such as Catch-22,
24-7, 5th, 7-Eleven etc.
Remove duplicate lines AND Count duplicate lines filters now do not require
that the duplicate lines be adjacent (i.e. that the list is pre-sorted).
However, it now has the restriction that it cannot speedily remove or show
duplicates from files LARGER than main memory.
Grep filter has now been changed so that it does NOT match empty lines.
TextPipe is now much faster when logging is disabled.
TextPipe no longer records output file names beyond a count of 10000. These
filenames are only used for .BAK file recovery, and they cause a performance
slow down when millions of files are being processed.
Fixed function for removing empty output files. The input file was being
deleted if the output file had zero size.
Fixed problem with duplicate data appearing in large sorts requiring merge
files with 2 or more merge stages.
Reverted handling of blank fields/blank lines to pre 8.3.5 handling. Pad to
Width and Add Lef - Margin filters cannot be used for blank fields (use an Add
Footer filter to give a blank field some width first).
Fixed bug in HTML Restriction filter where the action of the 'Include start
and end tag' option was inverted.
New Simultaneous search replace option for Filters\Search\Search and Replace
List Filter- use when your search and replace terms overlap each other e.g.
replace 1->7, 7->10, 10->1.
Copybook filter now recognises field names starting with numerics e.g. 14B-
DATE.
Copybook filter now copes with nested OCCURS, and numbers the generated field
names correctly.
New 'Copy to Clipboard' option when a hex dump is displayed from a field's
context menu.
New Output filter option to remove empty output files (those with a size of 0
KB).
Fixed performance of Trim option of mainframe copybook filter.
Fixed handling of blank fields when using Restriction Filter and a subfilter
such as Pad to Width, or Add Lef - Margin.
Fixed file modification detection when using subfilters of a Search/replace
or HTML Restriction filter.
Fixed memory leak that occurred when the Skip Binary files option was used.
Fixed VBScript/JScript code generated - missing end of line for
TPWindow.inputDeleteFiles.
Fixed bug with [Delete BAK files] button not being enabled.
Installer changes to support Vista better.
Fixed bug with Converting Excel spreadsheets to text when the output was set
to the clipboard.
If logging is enabled, a setting of 'Don't log to file if line count is less
than' = 0 now causes all lines to be logged, rather than none. To disable
logging, uncheck the logging flag. This primarily affects calling filters
from the TextPipe Engine.
The copying of all predefined global variables from the command line
(@name=value) now occurs no matter how the filter is called.
Enhanced Windows Explorer Context Menu handler to better handle huge sets of
files (20,000+).
Fixed bug in using UNC paths \\server\share etc - these conflicted with the
environment variable parser's desire to eliminate duplicate \\
Changed Directory scanning routines to allow files with any attributes.
Fixed bug in using external reference file for Replace->Search/replace list
Now recognizes the VALUE clause in mainframe copybooks.
EasyPatterns CSVfield, TABfield and Pipefield now handle unquoted fields
correctly.
Macros can now include environment variable values e.g. @ENV_OS inserts the
text 'Windows_NT' on an NT system.
Filenames specified in TextPipe can now make use of embedded environment
variables using e.g. %PROGRAMFILES%, %TEMP% syntax.
- Search/replace list filter
- Log filename
- Execute filter input/output filename and command to run
- Sound to play on completion, and default editor
- Add Left/Righ - Margin, Header/Footer, Add Text Side by Side
- Output filter new extension, new output folder and single output
filename
- Merge and Link filters.
New Output Filter option for No Output - the filtering is used for its side
effect only.
Now recognizes the new Office 2007 format .xlsx (e.g. search/replace lists).
Fixed line count statistics displaying -1 for subfilters that are never
invoked.
Default is now to process compressed files (.zip, .docx, .xlsx, .pptx).
Restored play sound on completion for command line filters.
Removed playing a sound in non-interactive mode (from command line or from
COM).
Interactive error messages are now disabled when TextPipe is called from the
command line or from COM (when TextPipe.visible = false).
Removed file copy on first run from Engine version.
New option to allow/prevent processing of files inside Compressed files such
as .ZIP, .DOCX, .XLSX and .PPTX.
[Copy lines to log file] button now also adds a log message to make the log
file easy to open.
Right-clicking the log results now shows an option to copy the log lines to
the clipboard.
Clearer warning for interactive logging ending.
Fixed EasyPatterns for CSVField, TabField and PipeField to avoid stack
overflows.
'Include the start and end tag' option of HTML filter was reversed.
'Output Filter\Open output on completion' now opens all output files, not
just the last one.
Better handling of copybook Comp3CanBeUnsigned option.
Now traps numeric conversion errors when sorting floating point values such
as 1e-9999.
Word concordance now has correct default for the output format, and the found
word is now correctly positioned in the context, and capitalized.
Fixed buffer underrun problem with search/replace filter, where maximum match
sizes larger than 4KB were not correctly performed.
New 'Paste File List from Clipboard' button on Files To Process tab.
Files to Process grid is now refreshed after loading a File List.
Embedded Help is now loaded correctly from the application folder.
URLs can now be inserted or loaded from a File List without being interpreted
incorrectly as filenames.
Fixed flushing of T-filter child filters.
Added new 'T-filter' to copy the data and process it in a different branch
without affecting the main branch.
Changed Unicode filter Unicode to Escaped ASCII to also convert Latin (128-
255) characters to escaped \Uxxxx format.
Improved right-clicking in filter tree to first select the node that is
right-clicked on.
Improved handling of invalid zip files.
Fixed problem loading replace filters from versions 7.8.3 and older with \@
or \$ in the replacement text.
Added new filter to Restrict to UTF-8 files.
Fixed bug with named parameters in search/replaces - if a subexpression was
not captured then any remaining named parameters would have no value and
corresponding global variables would not be set.
Clipboard output now shows a non-zero size.
Statistics are now reset correctly when a filter has subfiters.
Copy to log button now correctly re-enabled when a job starts for the second
time.
Improved internal patterns used by the HTML filter to prevent stack overflow
errors.
Fixed bug where /Q and /MINIMIZED were not recognised when placed in a /Z=
command file.
Fixed bug where subfilters of Restrict to Matching lines were not called when
a count of matching lines was being output.
Fixed discrepancies between docs, TextPipe and TextPipe Engine for
addSecondaryOutputFilter() function, output options and for filenames in the
File Grid.
Fixed error using a Search/Replace action of 'Send variable X to subfilter'
when the previous replacement text had been set to '$0'.
Improved handling of non-DOS EOL files - TextPipe now checks if your filter
has an EOL Conversion filter before offering to add one for you.
Link filters can now use a path relative to the current filters path
(previously this was relative to the TextPipe installation path). This makes
them far easier to link to.
The Capture Filter now allows variable names containing digits.
Globals can now be set on the command line using the syntax @name=value with
a space between each set of values.
New extended logging for Remove Blank Lines, Remove Blanks from Start/End of
Line.
New notice level message if MS Word or Excel files are processed using the
evaluation version.
If Remove/Restrict Delimited Field filters remove fields from the end of the
line, TextPipe now removes any hanging delimiters.
The Convert Excel files to Text filter now converts all sheets, not just the
first one.
Zone - decimal conversions now allow positive and negative signs.
When run from the command line, TextPipe now does not show a dialog on errors
such as a filter file could not be loaded. An exit code is set and the log
file written instead.
Fixed a bug where the Analyse function on the Files Grid did not return a
result.
Fixed a bug where the Copybook Filter used the wrong parser if PIC was
preceded by a tab instead of a space.
Copybook Filter now copes with 'PIC GROUP' syntax.
Capture filter can now resets at the beginning of the file, at the beginning
of each region or both.
Right-clicking a filter now shows a 'Help on this filter' link which quickly
takes you to the help file without needing to add the filter. Using
Help\Embedded Help also constantly updates to show you help on the current
filter.
The Status tab is now displayed by default when filters are run from the
command-line or from Windows Explorer.
Mainframe filters now insert implie - decimals correctly when the fields are
blank padded.
Folder overload prevention for the split filter now triggers at 1000 files
(was 10000).
Now warns if the user tries to use the Trial Run area to process a Mainframe
Copybook (a file must be used).
Task Menu filters now open a status window so that progress and/or errors can
be reviewed.
The evaluation version (even an expired version) can now be used to check if
a filter is compatible with the Pro, Standard or Lite version.
The Restrict to Filename filter (used to rename files on the fly) now skips
processing filenames inside zip/docx/xlsx/pptx files.
Fixed implie - decimal inserted in wrong position for zone - decimal fields.
The Status Window now forms a new tab of the Filter Window - making it easier
to refer to results. The results log is now also far easier to read.
The Mainframe Copybook Filter (new) and Wizard (deprecated)
- now handle OCCURS without TIMES when the repeating field is an unnamed PIC.
- now handle masks like PIC 9999/99/99 - these are treated just as an Alpha
field, in this case, PIC X(8). No mask substation is performed.
The Mainframe Copybook Filter now generates numbered field names when the
fields are inside a repeated group. This wizard does not do this, but the
field header can be manually edited.
Improved the EasyPatterns CSVfield, TABfield, PipeField, HTMLtag,
HTMLstartTag, HTMLendTag, QuotedString so that they are much faster.
Fixed bug with Search/Replace in Exact mode, if the Action was set to
'Remove', the text was not actually removed.
Added collection and logging of statistics to Lite and Standard editions.
Log filenames are now displayed with an absolute path to make them easier to
locate. When [Copy Lines to Log File] is clicked in the Status Window, now
ALL lines are copied.
Fixed handling of invalid .zip files.
Convert MS Word/Excel documents to text - better handling of MS Word/Excel
failures.
Improved status dialog with single tab display.
Log files now combine date and time into single field.
Fixed output of implie - decimal values for Mainframe Copybook conversions.
Fixed binary file detection logic.
Fixed handling of last file in compressed Office 2007 formats (docx, xlsx,
pptx).
Fixed script generation for addUnicodeConversionFilter2.
Fixed COM function TextPipeFilter.compiledFilterStartJob.
Grep functions can now match blank lines using the pattern ^$.
Fixed Copybook Filter choosing incorrect parser if 'pic' used rather than
'PIC'. Copybook Filter now accepts field names that include spaces. Parse
tree output now includes periods so that it is still a valid copybook. Fixed
field width for P (Packed) type when using alternate style copybooks.
The File Input's Binary detection now correctly identifies Unicode files, Zip
files and Microsoft Office 2007 file formats (.DOCX, .XLSX, .PPTX) as non-
binary.
Debug Mode of Copybook Filter now shows field length and start and stop
columns. Copybook Filter now handles OCCURS after the PIC definition.
New internal editor for reviewing text files.
Zip files - now processes and updates data stored inside compressed files
such as .zip, and Microsoft Office 2007 file formats such .DOCX, .XLSX, .PPTX
etc.
Word conversion filter now clears previous results in case of error, and
disables macros during run.
Removed 64k limitation on string data saved to registry, fixed error with
string data over 64k not being allowed.
Fixed Width Wizard now accepts both
and
-
formats.
Now uses updated pattern match library to resolve regex problems.
New status window option to Delete backup files (in addition to restoring
backup files). Status window now warns you if you attempt to close it before
dealing with backup files.
Better warning message for invalid file patterns in the Files to Process
grid.
Copybook REDEFINE clause now allows an optional trailing period.
New Word Concordance filter under Filters\Convert.
New filter Restrict to Blocks of Lines - useful for adding text after every
XX lines.
Reinstated Filters menu for accessibility, and File Menu items Link to
Filter, Insert a Filter, Search for a filter.
Refined the pattern generated by filter Restrict to Between Tags.
Script filter now works with PerlScript engine - all function validation is
disabled.
Added semi-colon (;) as an output format for the Mainframe Copybook Wizard
and Fixed Width Wizard.
Added error for blank text in the Mainframe Copybook Filter.
Fixed incorrect length detection for signed zone - decimal in the Mainframe
Copybook Wizard (the Copybook Filter used the correct result).
Reinstated wizard on startup.
Fixed pattern generated by filter Restrict to Between Tags.
The default filter Normal.fll is now loaded from the Documents folder
regardless of which folder TextPipe is started from.
C:\ is now not used as a temporary folder, instead it is requested from
Windows.
Maps are now retrieved from the Documents folder.
Code page functions are now unavailable under Windows NT.
New prompt for language on first run.
New Multi-Processor support (in Server version) with Filters\Special\Switch
CPUs to direct filtering to different CPUs. This enables massive performance
enhancements for large data crunching tasks.
Vista install support.
Now detects Unix files being loaded into Trial Run area, and warns the user
and offers to add an End-Of-Line conversion filter.
Mainframe Copybook Wizard - fixed incorrect implie - decimal.
- has been completely re-written as a filter. The parser now shows the line
number of any parse error, handles INDEXED BY clauses, logs warning if
invalid record at end of file, XML output option.
- now handles non-PIC style copybooks
TextPipe now highlights the filter in error correctly, taking into account
comment filters.
New macro @filterFilename to output the filter filename.
TextPipe.filterFilename is also available in the scripting filter.
Comment filters are now shown in green italics to make the filter list more
readable.
Fixed pattern generated by filter Restrict to Between Tags.
An inability to log messages to the log file now causes a Windows Event Log.
Tightened up the Analyse File Format function to correctly identify BOM and
DOS End-Of-Lines in files.
Fixed intermittent /Q Exit On Completion switch. This only occurred on fast
CPUs when a new processing thread started and finished before (!) the main
thread processed the /Q command.
Fixed EasyPattern bracket optimization. Now shows the line number of any
error, and fixed [PhoneNumber] pattern.
Fixed Perl/EasyPattern exception with named captured variables and Prompt on
Replace was turned on, when the named captured variable did not match
anything.
Removed remaining memory Info log item on Merge Sort.
Mainframe Zone - Decimal expansion now allows negative to be coded with '2' or
'5'.
Comment and Scripting filters now get display of special characters turned
off by default.
Mainframe Wizard now removes the trailing delimiter on the last field.
Explorer Context Menu now shows the filename without the extension.
Convert Excel to CSV filter now does not use a TEMP file in C:\
Mainframe Copybook Wizard now
- correctly discards INDEXED BY statements.
- removes the FILLER header associated with removed FILLER fields.
- VALUE lines are now ignored rather than causing a warning.
Windows from TextPipe.Alert calls now allow the filter to be cancelled.
Fixed bug with handling of Replace Action of 'Send $x to subfilter'.
Fixed bug with HTML filter removing the Restricted text.
New 'Action' field for Search/Replaces, with the default being just
'Replace', as before. The new field allows the matched value to be Removed,
to be sent to subfilters, or send just part of the matched value, such as $1-
$9. It also allows NON-matching text to be subfiltered, making it very easy
to process text outside bracketed expressions
New Output Filter options:
- Only Output Modified Files has been extended to allow the original files to
be deleted if they have been changed. This is very useful for repeated
processing of files to different output folders.
- Ignore Initial Folder - when the folder structure is being maintained, this
allows the start folder to be removed from the destination folder name.
EasyPatterns now support named capturing using the syntax:
[ capture( ... ) as 'VarName' ]
Optimized the Head/Tail filter when Including Bytes from the End of a file.
The Filters Remove Lines From List and Extract Lines From List now do not
recognize special characters such as \. This is so that lists of filenames
with backslashes can be pasted in.
File Grid 'Ignore' option renamed to 'Disable'. File Grid now allows files to
be excluded by perl regex pattern or EasyPattern.
Replace List filter now allows Find/Replace strings to be Swapped, and the
list to be reversed - very useful if you apply changes to your data and then
reverse them at the end of the filter list.
Fixed bug with the Restrict to Filenames Matching Pattern filter
- the filename was not being passed to subfilters, so double-level
filename restrictions did not work
- the patterns were incorrectly unescaped before being parsed, so
patterns with backslashes were not correctly interpreted.
If too much data is sent to the Clipboard, TextPipe now traps the error and
limits the text to 10MB.
Fixed bug with Restrict to Lines in Pattern list - the patterns were
incorrectly unescaped before being parsed, so patterns with backslashes were
not correctly interpreted.
Added scripting functions TextPipe.fullOutputFilename, TextPipe.FilterWindow,
TextPipe.Application.
Added COM functions FilterWindow.Application, FilterWindow.outputFileCount,
FilterWindow.outputFile() and FilterWindow.outputFileTask().
Now warns against CSV or Tab data containing ASCII 26 when converting to XML
(remove or change using a map or Search/Replace filter).
Fixed EasyPattern for [EmailAddress].
Updated Extract\Email addresses filter.
Fixed bug in Match Between HTML tags - Match Case was being set to On rather
than using the displayed setting.
Increased maximum stack size to eliminate Stack Overflow errors.
Fixed bug in Capture Text filter where the new captured value was only set at
the end of a file.
Processing is now suspended when you hit [Cancel].
Added TextPipe.exe path to Run... path.
Fixed bug - the Mainframe Copybook Wizard no longer tries to replace
delimiters in the input file. It was replacing delimiters even inside packed
decimal fields.
Fixed parsing of replace expressions including \x24 (a $ sign).
Mainframe Copybook Wizard new options - Remove Filler fields, Include header.
The Wizard now records option settings in the Comment output.
Fixed sort function not working when start column <> 1 and length < 4096.
Extract email addresses now ignores '&'.
Loading a search/replace list from Microsoft Excel now allows Unicode values.
The grep filter now always reports the output file as being 'modified',
preventing confusing behaviour with the 'Only output modified files' Output
Filter option. A change of filename from input to output now also results in
a file being written, regardless of any changes.
Mainframe Copybook Wizard now handles PIC fields both before and after OCCURS
xx TIMES for single field OCCURS statements.
Optimized tree updating as field data is changed.
Changed install paths for Windows Vista.
New internal function TextPipe.finishedWithFile() to halt any further reading
of the current input file.
New TextPipe.Application function isBinaryFile() to detect if a file is
binary or not.
Updated memo editor and pattern matching engine to fix bugs in third-party
components.
Fixed EasyPattern handling of EBCDIC() function.
Fixed hex dump filter outputting the last line twice.
Fixed handling of Compress to EBCDIC Packe - Decimal.
Fixed bug converting old replace patterns of form $1$2$3 to new form of
$1$$2$$3.
New filter to Add a Line at a given line number.
Fixed bug with EasyPatterns not returning all of a captured expression.
Unicode conversion filter now makes use of the 'error' character.
Mainframe copybook filter now copes with INDEXED BY statements, and better
handles nested OCCURS.
Escaped Unicode now outputs \uXXXX instead of \u+XXXX.
New option to show special characters such as spaces, tabs and line feeds.
New option - triple-clicking a line selects the whole line.
New filter to convert Filters\Convert\UTF16-LE to Escaped Unicode (ASCII +
UCN \U) e.g. \u402d\u403d
The file list can now contain http://, ftp:// and gopher:// urls to download
directly.
The user is now given the option to manually start Word or Excel if they are
not running when TextPipe needs them (with the Convert Word/Excel Document to
Text filters).
Pattern matches can now name captured variables for use in the replacement
text e.g. searching for (?P\d+) would then allow the macro @dollars
to be used in the replacement, and also as a global anywhere else in the
filter list.
Far more complex patterns can be processed, with support for up to 99
captured variables, from $1-$99.
New 'File\Send to...' function for filter lists.
New Fixed Width to Delimited Output wizard - makes converting fixed width
files to CSV, Tab simple.
Mainframe Copybook Wizard now allows underscores (_) and single quotes (') in
field names in 'XXX_XXXX 4 AN 1-4'-type copybooks.
Wizard now removes the content of fields called FILLER*.
Wizard now copes with repeated single-fields, as well as GROUPED OCCURS.
EBCDIC packe - decimal fields can now have either a leading sign, trailing
sign or no sign.
New option to trim leading spaces and zeroes from fields.
New option to only expand numeric fields - Alpha fields are left unchanged.
This is useful when the target application can accept EBCDIC data but can't
handle packe - decimals.
New option to allow COMP-3 to be unsigned.
Wizard can now cope with removing line numbers and commented copybook lines
(those with a '*' in column 7).
Wizard can now cope with field definitions spread over multiple lines.
Wizard now remembers settings between uses in a single session.
Expand Binary Number to EBCDIC filter now outputs a fixed width result.
Expand Unsigned EBCDIC filters now output a space for invalid digits A-F.
EasyPatterns now produce optimized perl regex output. Added EasyPattern
EBCDIC('') for matching EBCDIC values. Fixed parsing error with ASCII()
function. Now handles back references to groups 1-99. Fixed DayNumber
insisting on leading zero. EBCDIC() and ASCII() functions can now be used in
conjunction with not ... as character classes.
Hex displays now all use '.' to replace space.
Query replace window now shows hex offsets for Hex Dump display.
Convert Word Document to Text/Convert Excel Spreadsheets to Text now support
Unicode output as an option. Follow this with Convert UTF-16 to UTF-8 to
process these files using normal filters.
New File Shred filter to completely overwrite file contents. Run repeatedly
for more shredding.
Unicode property tables for pattern matching updated to Unicode 4.1.0, and
matching engine updated to PCRE 6.4.
Split filter now can split at a pattern.
New Split Filter option to avoid overloading directories. When more than
10000 files are created, TextPipe starts a new folder.
Fixed bug where \\ could appear in split output filename.
Filter list tree now auto-scrolls.
Filter list tree now allows mass enable/disable of filters with select and
right-click.
New Filters to Add File Headers/Footers. Unlike the existing Header/Footer
filters, these filters only output once per file even when included inside a
restriction.
Now can drag and drop filters between filter lists in different windows.
Filters Add Header/Footer/Lef - Margin/Righ - Margin now always check for
@macros embedded in the text, regardless of whether the text comes from the
memo or an external file.
Fixed bug with search/replaces where an internal conversion from ansi to
Unicode and back could result in high-ansi characters not being mapped
correctly back to their original characters. In particular, the ansi (tm)
symbol mapped back to TWO characters.
@macros are now allowed in the replace field of a search/replace.
Note: If you load a filter from an older version, '@' will be escaped to
'@@'. The Confirm Replace dialog now shows the values of all global and user-
defined @macros.
Warning for Excel or Word documents now disabled.
Fixed Seller's Edit Distance search/replace not working.
Fixed bug in Mainframe Copybook Wizard for length of signed Packe - Decimal
fields.
No longer deletes current output file when processing is cancelled and the
file is being appended to.
TextPipe now works with DataPipe successfully.
To prevent accidental overwriting of files, the default path for the Output
Filter's Single File Output, and the Merge Filter path are now set to blank.
Removed the support for quoted tab data. Tab data does not need quotes!
Fixed error with deleting filters.
Fixed error entering custom delimiter when '\t' was entered.
Ultra-fast sorting of huge files (100MB in less than 30 seconds); control
over maximum memory used - for shared servers.
Convert Excel spreadsheets to CSV format.
Control which filters are applied to which file types with the new Filename
Restriction filter - apply filters to say *.doc files, or *.xls etc.
Handle more types of Mainframe data than ever before, with the improved
Mainframe Copybook Wizard:
- discarding line numbers from pasted copybooks
- now handles null packed and zoned numeric fields
- now outputs invalid digits in packed or zoned numerics as their hex
equivalents
- now adds filters to trim leading and trailing space from fields, and
leading '+' signs from numerics
- can now choose output format between Fixed, CSV, Tab- and Pipe-delimited
- now can add leading zeroes to numeric fields where the implied position is
greater than the field width
- output format now includes a record header, which makes it far easier to
validate the output file and import the file into a database
- new filters for unpacking odd/even-length unsigned packed numerics, and for
decoding overlapped unsigned packed numerics
- delimiters are removed from the input file first to prevent extra fields
being created.
Easily Extract Text Between Delimiters with the new Web-based Filter Wizard.
Review results easily
- New Output Filter option to open the output file on completion.
- Open output files easily with clickable links in the Status Dialog.
Set the Error Character for Unicode and Code Page conversions when there is
no appropriate character in the target encoding.
For precision automation, control TextPipe's pause before exit with the new
/Q=millisecond option.
Accept pasted Unix text in the Search/replace filter.
Easily match day names, day numbers and arbitrary characters using the new
EasyPatterns [dayname], [daynumber] and [ascii()] function which inserts
arbitrary ASCII codes in hex (eg $ff) o - decimal (e.g. 255) format.
EasyPatterns now accept counts and quantities in either hex o - decimal
format.
Numbers lines across files, with the 'Don't Reset At End Of File' option of
the Line Numbering filter.
Count occurrences of all lines, with the 'Include Counts Of 1' option of the
Count Duplicate Lines filter.
Debug the output files of External programs using the 'Remove Output Files on
Completion' option of the Execute Program Filter.
Store arbitrary text in the Scratch Pad - new Edit\Preference option.
Date and time macros now return consistent results across different
localities -
@time - Inserts the time in the format HHMMSS
@date - Inserts the date in the format YYYYMMDD
@datetime - Inserts the date and time in the format YYYYMMDDHHMMSS
When the right-click menu 'Save' option is selected, TextPipe now saves the
text exactly as is without interpreting special characters.
TextPipe now prompts if you try and exit while a thread is running, and asks
for confirmation if you click [Cancel].
When a list of perl patterns is loaded or used, TextPipe now shows the line
number of any error when a pattern parsing error occurs.
The grep form's context menu now displays EasyPatterns or perl patterns
according to the pattern type selection.
When new filters are added or duplicated they become selected. New filters
are now added with blank fields, and for the Search/Replace filter, the extra
options are set to defaults. After deleting a filter, the next filter is now
selected.
The Options panel of the filter tree now scrolls to display buttons that are
not visible.
Better handling of files that are missing or open in another process.
Fixed Range Check error with optimization of 1:1 maps.
Fixed Single File Output - didn't force creation of folders for output file.
Fixed Windows Explorer shell extension - was starting the filter one row down
from the correct one.
Fixed bug where Extract URLs filter would miss a URL on the very last line.
Fixed merge sorting bug (only for very large files) where processing never
ended, and duplicates were not removed.
New macro @fullOutputFilename - May be blank if the clipboard is being
processed.
Fixed deadlock between threads that was slowing down the processing when the
TextPipe.Application object was called from code.
Fixed Windows Explorer Content Menu not appearing.
Fixed bug with grep filters where Match Case option was not recognised.
Standardized help text and labels for grep filters.
Fixed bug with conversions between ANSI and ASCII.
Fixed bug with delimited field parser when it encountered a ' or " in the
middle of the field.
Fixed bug with clipboard button not being acted on correctly.
New filters to Restrict to ANSI files, and Restrict to Unicode UTF16 files
and Unicode UTF32 files. These are exceptionally handy for only converting
ANSI files to Unicode, or only converting Unicode files to ANSI.
New filter to convert PDF documents to text format.
Menu re-organization: Email menu replaced by new Extract menu, encodings
moved to Convert menu. 'Retain...' items renamed to 'Extract...'.
New macros @randomdigit and @randomletter - very useful for generating test
data or for generating random sort keys.
New Explorer extension option to Analyse files - detect if they are Unicode,
ANSI, etc.
Reverse File performance improvement.
New Unicode filters for Upper case, Lower case and Title case.
Speed improvements for handling very large files.
New Expand All/Collapse All menu on All Filters view.
New scripting function to build filters to convert a Mainframe Copybook.
Copybook wizard now inserts a warning for each unidentified field, making
incorrect fields easy to identify.
New warning message if no files were found to process.
Filters\Convert\Numeric HTML entities To Text filter now converts output
characters to UTF-8. The resulting text can then be converted from UTF-8 to
UTF-16 for display.
Menu items for common grep and conversion filters in the Wizard menu. Wizard
menu now is always expanded.
Memo and text fields now use the fixed width font.
Unicode and code page conversions now do not load the entire file into
memory, they convert the file progressively.
Updated Simplified Chinese translation.
Help text added to the Unicode/Code page conversion filter.
Moved Hex and HTTP encode/decode to the Convert menu.
Fixed pasting of text into a memo's Find window.
Drag and drop now prevents nodes being dropped on filters that do not support
subfilters.
Unicode replacement now reverses the order of Match Case to be correct.
Reverted help system to old style.
New Restrict to lines from list/lines NOT from list and
Restrict to lines matching pattern list/lines NOT matching pattern list.
New Match Case option for these filters too.
EasyPatterns for Month and MonthName now do not automatically capture.
Mainframe Copybook Wizard now copes with OCCURS where the number of
repetitions is fixed.
New filter to convert numeric HTML entities to text e.g. ·
. New
maps to convert text t - decimal or hex HTML entities.
/CLEARFILTERS now removes all filters except the Log, Input and Output
filters. This retains the default settings from normal.fll
/EXTENSION can now be set to blank to disable changing the extension.
Fixed Access Violation processing empty result sets.
Fixed load file list function.
Now shows all available filters in - separate tree - these can be drag and
dropped or double-clicked to move them into the filter tree.
Fix to Mainframe copybook filter to allow it to recognise field names that
included digits followed by ' A'.
Fixed bug with Unicode exact match and Unicode pattern match where extra
characters were being output.
Fixed bug with Unicode exact match and Unicode pattern match where search
terms were not found.
Mainframe copybook wizard is now more flexible in the data it matches, and
handles COMP-3 field types, and handles numeric fields with purel - decimals
(eg V999).
New filters for code page conversion.
New filters for Unicode Compose an - Decompose:
NFC - Canonica - Decomposition, followed by Canonical Composition
NFD - Canonica - Decomposition
NFKD - Compatibilit - Decomposition
NFKC - Compatibilit - Decomposition followed by Canonical Composition
And Canonical Compose.
Now supports Unicode command line parameters.
Now supports single quoted CSV data, quoted tab data, and backslashes in
quoted CSV or Tab data. Existing text delimiters are now preserved on
conversion between CSV and Tab formats. The EasyPattern [tabfield] can now
match quoted tab data.
The Restrict to HTML filter can now set the maximum match size - to match
very large sections of HTML or XML.
Clarified examples in perl pattern right-click menus.
Speed optimization for interactive search/replace when [Replace Job] chosen.
When files are dropped, TextPipe defaults files to 'no subfolders'.
Simplified the Trial Run button - now it only performs a trial run, and does
not toggle back to the Filters to Apply tab.
XSLT filter now correctly flags modified text - so output files get created
when the 'Only Output Modified Files' flag is set. XSLT filter now reports
the error code and error reason to the log if an error occurs.
Run External Program filter now reports in the log if the external program
has a non-zero exit code. The output file handle is now released on error.
The Unicode Map filter's 'Replace With Value' string now supports special
characters.
Encode Hex filter now does not add line feeds.
Right-click option Show Hex Dump now checks if the underlying field is
Unicode or not before displaying data. A toggle between UTF16-LE and UTF16-BE
is also available.
Reduced memory requirements of Database filter when SELECTing data sets.
Fixed database filter's handling of VarBinary (BLOB) data.
Database filter now handles subfilters correctly for encoding VarBinary data.
Fixed bug when referencing a CSV or Tab search/replace list where the
replacement text was missing. A new error message now provides the line
number of any errors.
Fixed bug with inserting captured text as an HTML table row, and added the
ability to insert as an XML record.
Fixed bug with HTML help crashing TextPipe on exit.
Fixed bug with Input Filter, Prompt On Binary files setting, if 'No To All'
was chosen TextPipe skipped ALL remaining files, not just binary files.
Fixed Unicode display of extended information in the Prompt on Replace
dialog. Note: Unicode patterns do not support capturing- only $0 works.
Fixed Map filter not detecting Reset and Clear changes.
Fixed help file opening twice when F1 pressed.
Fixed mismatch of 16 and 32-bit Unicode strings when converting from ANSI or
ASCII/OEM to Unicode and between code pages.
Restored registry settings required by Offline Explorer.
Changed default for re-loading previous filter to Off.
Better exception handling for XSLT filter.
New Mainframe Copybook wizard makes it incredibly easy to build a filter to
process mainframe data. Handles packed, zoned, binary and display field
types.
New filters to Expand Binary Number To ASCII and Expand Binary Number To
EBCDIC.
Global dialogs now make use of popup menus to insert special characters etc.
New filter to capture text to a global variable. This is much faster than
using the Script filter to capture and then output text - which is the most
common use of the Script filter.
New grid component for the Map filter - now supports far better range and
mouse selection.
New option to automatically reopen the last filter.
The Replace Query dialog now has an extra tab to show a hex dump of the found
text. This can be useful for debugging.
Right-click menu for Replace field now provides easy insertion of captured
data as Tab or CSV-delimited fields or HTML.
The Prompt On Replace dialog now displays the length of each captured
expression.
EasyPatterns 1.9.1 - added [PipeField] and [Pipe] keywords.
Fixed error when manually choosing the Skip option to not replace text in a
DBCS file - more than one character was added. Only occurs under DBCS
versions of Windows (eg Chinese).
New error message when there are too many capturing expressions in a pattern.
Fixed bug with 'First row is header row' not being recognised for processing
delimited data with embedded carriage returns.
Added short cut menu items for converting between Unicode/ANSI and
Unicode/ASCII.
Added line number and offset properties to the TextPipe object available in
the scripting filter.
Map filters now have a comment property to describe what they are doing.
EasyPatterns 1.9 - added keywords for [number, numbers, leftAngle,
rightAngle, lessThan, greaterThan ]
Fixed restriction filters not supporting Secondary Output filters properly.
Fixed exact match filter not properly supporting Extract option, and not
correctly handling text when 'Skip' was chosen.
Search/replace lists now detect inserts, deletes and moves as changes. The
order of moved rows is now retained.
Fixed restriction filters not supporting Secondary Output filters properly.
Fixed EP keywords for [CRLF], [DOSNewLine], [EmailAddress] and [Hyperlink]
Fixed width font for Trial Run Input/Output is now correctly loaded.
EasyPatterns now allow repetition to be specified using the forms [2..3 ...]
and [2-3 ...] in addition to [2 to 3 ...].
Right-click menu item Escape Special Characters now works for EasyPattern
fields.
Fixed format string error on filter export.
Fixed Search/Replace list -caused an error when it was opened.
Fixed Remove Byte Orde - Mark (BOM) filter when handling UTF-8 files.
Added right-click menu to EasyPattern fields to make designing patterns
easier.
Added Line, Column and Length indicators to Search/replace filter.
A warning is now shown for trailing whitespace in the search text, which
often results in the text not being found.
Added 5 second countdown before TextPipe exits due to /Q command line
parameter.
Greatly improved performance for massive numbers of files- TextPipe no longer
checks for duplicate filenames.
Fixed EasyPattern for [month,monthnumber,day,hour] so that they match the
long version of the pattern first, then try the shorter variations.
Fixed range check errors with Compress packed/zone - decimal filters.
Fixed EasyPattern for [word] - previously matched non-word characters.
Fixed EasyPattern - '.' was not escaped - now fixed.
The Prompt On Replace dialog now shows the perl equivalent to an EasyPattern,
for verification purposes.
Fixed access violation with changes to the Restrict to Bytes filter.
Improved Restrict to Bytes (Byte x to End-y) to handle unlimited amounts of
data
You can now set the field delimiter for the Restrict To Fields, Copy Fields,
Move Fields and Delete Fields filters. This opens the way for semi-colon-
delimited, #-delimited, !-delimited, ^-delimited and other bizarre data
formats.
Added two new filters for compressing mainframe EBCDIC data to Packe - Decimal
and Zone - Decimal formats.
The Split filter can now set the first filename number.
Added new Insert Bytes filter to complement the Insert Columns filter. This
new filter disregards line feeds when it adds text, and is very useful for
inserting commas into EBCDIC text as the text is converted.
New documentation links in the Filter Menus.
Synchronized scrolling is now properly restored when a filter is loaded.
EasyPattern fixes to align with the EasyPattern documentation
- [symbol, symbols] - removed the '!'
- [space, spaces] changed to just matched ASCII 32, not tabs, line feeds etc
- [horizontalwhitespace, hspace] - now only match space and tab
- [verticalwhitespace, vspace, worddelimiter, linedelimiter,
paragraphdelimiter] - now match formfeed
- [anybracket, anybrackets] - now match { and }
EasyPatterns now will NOT attempt to combine OR'd negated and non-negated
character sets, such as [ not lineChar or 'a' ]
HTML filter now matches tags when the tag name is immediately followed by a
tab or newline, not just a space.
Fixed EasyPattern handling of vertical tab (\v), which affects the keywords
verticaltab, verticalwhitespace, vspace, worddelimiter, wordchar,
columndelimiter, columnchar, linedelimiter, linechar, whitespace, word, line,
and column. This problem still exists in the perl pattern library, and can be
worked around by using \x0b instead of \v.
Expand Zone - Decimal and Expand Packe - Decimal filters are no longer added
inside a Restriction filter.
Changed EasyPatterns for EmailAddress, HyperLink, PageNumber and DayOfYear so
that the constituent parts are captured into $variables for later
substitution.
Fixed EasyPattern for FormFeed.
Fixed handling of EBCDIC negative zone - decimals.
Fixed bug with parsing of text with \x{...} characters.
Fixed shell context items - the last item did not work. Also fixed the path
for the Split File filter.
Fixed bug with Rolling back .BAK files - if the Only Output Modified Files
option was on, it would delete original files.
Unicode conversion filter was only outputting the Byte Orde - Mark for the
first file.
Added swap button to Unicode conversion filter.
Fixed Unicode conversion filter bug.
Fixed Backup Mode behaviour when the .bak file already existed (now it is
deleted).
Fixed Hex Dump option for Edit Fields and File Grid.
New Unicode (UTF-16LE) search and replace, with options for space compression
and for ignoring non-spacing characters.
New Unicode (UTF-16LE) pattern matching.
Unicode conversion filter is now much simpler, as you just choose an input
and output format in one step.
New XSLT filter for manipulating XML.
New file grid right-click menu items:
- Analyze file - checks for various Unicode, XML, CSV and Tab delimited
formats, Unix/PC/Mac end of line format and other file types
- Hex dump - displays a hex dump of the first 200 characters of the file
Added \x{...} Unicode character matching for exact matches. This means that
for a Unicode space you can type \x00000020 instead of \x00\x00\x00\x20.
Synchronized scrolling option for Trial Input and Output.
You can now control if the Trial Input and/or Output are treated as Unicode.
Optimized single byte maps for the special case of one output character for
every input character.
Better help contexts for the 9 different search/replace types.
Removed restriction on pattern recursion depth - this was causing Stack
Overflow errors for legitimate expressions.
Fixed file grid ordering - if you moved rows around, TextPipe did not
remember the new row position.
Fixed menu items for Remove, Copy and Move tab fields. These did not add the
required filters.
Fixed help hyperlinks on Trial Run and Files tabs.
Fixed bugs in Unicode Make Little/Big Endian filters.
Fixed bug in Convert From Unicode filter, for when it encountered a character
that could not be converted. It now outputs a space.
The EasyPattern [year] keyword now checks for a 4 digit year before checking
for a 2 digit year.
Convert Word Document To Text filter now only closes the Active Document,
avoiding the MS Word error message 'This method or property is unavailable
because a document window is not active'.
Fixed line wrapping problem with the clipboard export text of several
restrict-type filters.
Increased maximum stack size to cope with increased size of Unicode filters
(and avoid Stack Overflows).
Fixed database filter not quoting char or date/time data fields.
New filters for handling pipe-delimited files:
Remove pipe fields
Restrict to pipe fields
Copy pipe fields
Move pipe fields
Added EasyPattern keywords for slash, colon, hash, pound, percent, star,
asterisk and ampersand. EasyPatterns now generate much better perl regexps.
EasyPatterns can now set the maximum expected match size - this allows them
to match text larger than the default of 4K.
Increased processing speed for Trial Runs and Clipboard output.
Restrict to fields filters now export the settings for Process Fields
Individually, Exclude Delimiter, Exclude Quotes.
Fixed EasyPatterns [month] and [monthname] to attempt to match a long month
name (January - February) first, then a short month name (Jan - Feb).
Fixed bug with Context Grep with lines after the match. Matching lines at the
end of the file were being omitted.
Fixed code for handling CSV, Tab and Pipe delimited files with header lines.
Added new EasyPatterns for [ CSVfield ] and [ TABfield ].
New Restrict to Byte Range filter - useful for expanding mainframe packed and
zone - decimal fields.
New Delete Byte Range filter.
XML filter now has better matching of double and single quoted attributes.
These filters now handle embedded double single or double quotes, quotes
escaped with backslash, and embedded new lines.
Optimized expand packe - decimal and expand zone - decimal for speed.
Fixed bug with swap UTF-16 and swap UTF-32 word order.
Added Expand packe - decimal and Expand zone - decimal filters.
Fixed handling of Restrict to between tags filter - under some conditions the
trailing > of the start tag would be put in the replace result.
Fixed Copy CSV/Tab fields bug. When copied fields were appended, the
delimiter was not inserted if the copy position was the next available field.
Added Unicode functions
- Remove BOM (Byte Orde - Mark)
- Swap UTF-16 word order
- Swap UTF-32 word order
- Make Little Endian
- Make Big Endian
Added 144 Unicode conversion functions to convert to and from:
ARMSCII-8 ASCII BIG5 BIG5HKSCS
C99 CES-BIG5 CES-GBK CNS-11643-1
CNS-11643-15 CNS-11643-2 CNS-11643-3 CNS-11643-4
CNS-11643-5 CNS-11643-6 CNS-11643-7 CNS-11643-INV
CP1046 CP1124 CP1125 CP1129
CP1133 CP1161 CP1162 CP1163
CP1250 CP1251 CP1252 CP1253
CP1254 CP1255 CP1256 CP1257
CP1258 CP437 CP737 CP775
CP850 CP852 CP853 CP855
CP856 CP857 CP858 CP860
CP861 CP862 CP863 CP864
CP865 CP866 CP869 CP874
CP922 CP932 CP949 CP950
DEC-Hanyu - DEC-Kanji EUC-CN EUC-jisx0213
EUC-JP EUC-KR EUC-TW GB18030
GB2312 GBK GEORGIAN-ACADEMY GEORGIAN-PS
HP-ROMAN8 HZ ISO-2022-CN ISO-2022-CN-EXT
ISO-2022-JP ISO-2022-JP1 ISO-2022-JP2 ISO-2022-JP3
ISO-2022-KR ISO646-CN ISO-646-JP ISO-8859-1
ISO-8859-10 ISO-8859-13 ISO-8859-14 ISO-8859-15
ISO-8859-16 ISO-8859-2 ISO-8859-3 ISO-8859-4
ISO-8859-5 ISO-8859-6 ISO-8859-7 ISO-8859-8
ISO-8859-9 ISO-IR-165 JAVA JIS_X0201
JIS_X0208 JIS_X0212 JOHAB Johab-Hangul
KOI8-R KOI8-RU KOI8-T KOI8-U
KSC-5601 MacArabic MacCentralEurope MacCroatian
MacCyrillic MacGreek MacHebrew MacIceland
MacRoman MacRomania MacThai MacTurkish
MacUkraine MULELAO-1 NEXTSTEP RISCOS1
SHIFT-JISX0213 SJIS TCVN TDS565
TIS-620 UCS-2 UCS-2BE UCS-2-Internal
UCS-2LE UCS-2-Swapped UCS-4 UCS-4BE
UCS-4-Internal UCS-4LE UCS-4-Swapped UTF-16
UTF-16BE UTF-16LE UTF-32 UTF-32BE
UTF-32LE UTF-7 UTF-8 VISCII
New context menu item to insert control characters like tabs and form feeds.
New context menu item to show a hex dump of the selected text.
Added logWarning to scripting model.
Added Cut/Copy/Paste to file grid context menu.
Added Tile function to Window menu.
The Convert Word Documents To Text filter now disables Word's screen display
for faster processing.
The TextPipe Engine DLL now has a COM interface identical to TextPipe's, but
removes most GUI operations for incredible speed.
Added alert() function to the internal TextPipe object of the scripting
filter, so JScript functions can easily display debug dialogs.
EasyPatterns bug fixes and improved flexibility. In particular, the [char]
keyword was fixed.
Several filters that should have allowed subfilters did not. Filters affected
were: database filter, insert column filter, add lef - margin filter.
Fixed bug - the filename was being output even for non-matching lines, when
context lines were included.
Fixed bug with partial trial run menu items including too many items.
Fixed bug with the filter tree allowing nodes to be pasted on the Input
Filter, resulting in error-producing subfilters of the Input filter.
EasyPatterns - English-like text patterns for mere mortals e.g. to match
phone numbers, use
[3 digits, punctuation, 3 digits, punctuation, 4 digits]
As easy as that!
Many fantastic new HTML and XML filters!
Restrict a search/replace or other filters to an XML element, attribute or
to text between XML tag pairs
Remove an XML element, attribute or XML tag pair.
New Restrict To Filename filter. This allows you to search and replace inside
filenames, number and rename files and much more!
Very powerful new debugging feature - selective trial runs. You can select
which filters are being tested without having to clone a piece of the filter.
You can perform a Trial Run on:
Entire Filter List (current behaviour, default)
Selected Filter Only
Selected Filter and Sub-Filters
Filter List up to and including Selected Filter
To see these new options, right-click on a filter in the tree view.
New Backup Mode - renames original files to .BAK. Easily restore originals
with a single click in the Results window.
New Convert Word documents to text files filter.
Fixed bug when the Result window Cancel button was clicked twice in a row.
Fixed bug with pasting filter tree items that were copied from level 2 or
deeper.
Fixed bug with the Windows Explorer Extension that showed the TextPipe menu
multiple times.
Fixed bug with multiple consecutive Script filters not passing header text.
Fixed bug with Search/Replace Enable UTF-8 option and Extract option being
swapped when saved.
Fixed loading of Unicode maps from .TAB and .CSV formats - blank fields were
erroneously skipped.
Fixed bug with Remove Duplicates option of Sorting filter not being
respected.
New macros @time, @date and @datetime.
Fixed bug with handling of CSV fields with more than one line embedded in a
column.
Vastly improved sorting performance for files larger than 50 MB.
Simplified user interface by removing
1. Comments tab (old comments are now inserted into the filter list as the
first item)
2. Forcing the MDI child window to initially appear maximized, making it look
like a SDI application. The window size can still be changed as before
3. Removed the title that appeared above the properties
4. Removing the scratch pad tab unless a loaded filter has text in the
scratch area.
Added UTF-32 Unicode Maps
When Prompt On Replace is on, it can now be disabled during the search. Note
- this does not update the tree view immediately, and if the filter is
currently selected it has no effect.
The evaluation version provides a warning if the user attempts to modify
Excel or Word documents.
Unicode maps can now append multiple map files together into one map.
Unicode maps can now log overlapping regions as warnings or errors.
Improved Unicode map timings for populating with Default or blank values.
Added multi-maps for mapping input Unicode to multiple output Unicode forms
depending on command line parameters.
/MINIMIZED now examined before any other parameter, eliminating flicker.
Fix error when the file grid was selected in the askSam version.
Fix for Run Time version that prevented filter additions after a filter had
been loaded with /F=.
New File Grid context menu option to copy 100 Kbytes to the Trial Run Area.
Upgraded to PCRE 4.3 pattern matching library to fix assertions not working
correctly.
Added warning message that not all filter types can be exported to the
command line.
Tips and Wizard dialogs now allow filters being double clicked on from
Windows Explorer to be loaded.
Tips and Wizard are only shown if a filter hasn't been automatically loaded.
Run Time version now disables sound by default and does not show a tray icon.
Filter overlay images are now updated correctly when a disabled sub filter is
deleted.
Added Exit Codes depending on the highest level of error logged.
New /EXPORTFILTERTOLOG command line option.
Changes to support TextPipe Engine file processing.
Trial Input now preserves Unix EOLs when a file is pasted or loaded.
Changed menu structure to make it easier for new users. Removed some icons
from the toolbar.
Fixed Access Violations with Remove Lines Matching Perl Pattern filter, and
on some random occasions, with Perl pattern matches.
Fixed code generation of setPerl() statement.
Fixed an extra blank line being inserted at the start of the next file after
a Unicode file was processed.
New filter supporting search/replace lists. The lists can either be specified
in a grid, or they can be loaded dynamically from Excel, CSV or Tab-delimited
file formats at run-time.
The file list now supports multiple file specification - separated by semi-
colons eg c:\INetPub\*.html;*.txt;*.htm
The Split Files filter can now control how the output files are named.
Subtract entries in the file list are now shown in red, and Ignore entries
(comments) shown in grey.
The Script filter now uses a Syntax highlighting editor to display VBScript,
JScript, VBA, PerlScript and PythonScript code. It also allow - separate
default scripts to be saved for each language.
Unicode maps can now be loaded from and saved to Excel/CSV/Tab formats.
Unicode filters are now far faster and can deal with undefined characters -
such as removing them, replacing them with a custom string, sending them to a
VBScript sub filter for specialized processing, or allowing them to pass
through unchanged.
The filter tree view now supports Cut, Copy and Paste clipboard operations.
There is also an undo action for the current filter. Filters can also be
pasted into any application as a text export.
CSV and Tab-delimited files with CR/LFs embedded in fields are now handled
correctly. New restriction filters can be used to remove the embedded CR/LFs.
CSV and Tab-delimited restricted filters can now remove the comma or tab
field delimiters and the surrounding quotes to make it easier for sub filters
to work with field data. Each field's data can now be processed individually
rather than as a concatenation of field data.
Byte map lines where the output is different to the input are now shown in
yellow. Byte maps can now scroll directly to the character you press on the
keyboard - which makes finding mappings easier.
In the audit log, merge file filters now summarize the changes made by
filters that follow them in the filter list. Split file filters also
summarize the changes made.
GUI and internal changes to support the new French translation.
New command line options /CLEARCOMMENTS, /CLEARFILTERS, /SAVEFILTER,
/LOGENABLE, /LOGMODE, /LOGTHRESHOLD, /PREPAREFILTERFORDISTRIBUTION,
/SAVEFILTER, /PLAYSOUND.
When loading a list of commands from a command file with /Z=, comments can now be included in the file using a leading ';'.
Speed improvements for the GUI and for filter execution.
Audit logging now shows status for more filters including Scripts and Maps.
The audit log text shown for search/replace filters now matches what is shown
on the screen for ease of reference.
Various COM scripting changes to support new features, and to allow DataPipe
to set the current directory.
Perl patterns now support UTF-8.
Convert end-of-lines filter now handles Mainframe CMS V- and F-format files
(downloaded to PC in EBCDIC format).
Removed Rich Edit Memo controls - they caused RTF codes in plain text to be
interpreted as fonts, and incorrectly added extra CR/LFs to Ad - Margins-type
filters.
The file list had a bug where it added file entries more than once if they
matched multiple file specifications in the same group.
Fixed bug with loading of single byte maps where the map was not actually
loaded, from both the menu and from the map sub-form. Fixed Reverse function.
Unicode Maps - fixed problem where changes to the grid were not detected and
saved.
Fixed bug with importing Excel and Tab-delimited search/replace lists.
Fixed bug with appending filters from the command line with multiple /F=
appearing in reverse order. Fixed filters such as /Hx not being added.
New filters - CSV to Tab-delimited, CSV to XML, Tab-delimited to CSV, Tab-
delimited to XML.
Fixed error message when egrep pattern matcher encountered a \000 (null).
Only perl patterns can handle nulls.
Search and replace lists can be loaded easily using the Load List from
Excel/Tab/CSV formats in the Replace menu.
The Task Bar Menu and Map Menu now display sub folders of the TaskMenu and
Maps folders, and sort all the filenames and folder names shown. This makes
it very easy to categorize these maps and filters into folders.
If a filter has one or more of its sub filters disabled, a red slash is shown
through its icon in the tree view. This makes it easy to find filters where
disabled sub filters are hidden from view. A disabled filter still has a red
cross through its icon in the tree view.
Custom patterns can now be defined in the file custom_patterns.txt in the
TextPipe.exe folder. The patterns can be added through the right-click menus
on every field.
Sorting can now handle files of unlimited size. Previously sorting was
restricted to using physical and virtual memory.
New sorting options for date and time, date, and time.
New right-click option for filter tree view to turn a filter into a sub
filter.
Matching/non-matching lines and other grep line based filters can now use
Perl patterns, egrep patterns, MS Word patterns and Brief patterns. The
addGrepFilter COM function has changed to support this.
Export to VBScript/JScript/web page now also exports the search replace
options such as greedy matching, allow comments, buffer size etc. The edit
distance threshold parameter has been removed from the addReplaceFilter COM
function, and new functions added for setPerl, setEditDistance and
setBufferSize.
Menus and tabs re-organisation, selection filters renamed to restriction
filters.
A fixed width font is now used for most memo fields.
Improved handling of malformed CSV files.
Fixed export of Logging:Append to Log. The wrong value was being exported.
Fixed generation of random files of specified size. All files were being
truncated at 8k.
Added "Content Type" to system registry to help web downloads of .fll files.
COM object now registers itself in the Running Object Table (ROT), and frees
itself at the correct times.
New COM properties Application.visible and Application.activeFilter
Fixed bug with moving nodes with sub filters.
The Task Bar menu now correctly runs filters for the second and subsequent
runs - previously the status window stopped it from working.
New filter to Remove a line range (in addition to the Head/Tail filter).
New filter to Remove CSV fields.
New filter to Remove Tab fields.
New filter to Move CSV fields.
New filter to Move Tab fields.
New filter to Move Columns.
New filter to Copy CSV fields.
New filter to Copy Tab fields.
New filter to Copy Columns.
File/Open command can now open multiple filters at a time.
Drastically improved the speed of 'head' filters and Remove All - they now
skip reading the remainder of the file no matter how large it is (a 'Head'
filter is used to extract the first N lines of a file).
New option to display help in an embedded window that is synchronized with
the current action.
Added filter export to VBScript and JScript - this can be used to very easily
create code for ASP or web pages or simple scripts.
Added COM interface to create Head/Tail filter, Run External Program filter,
Split filter, Show/Remove Duplicates filter, Secondary Output filter. Logging
functions, Input filter and Output filter settings can now be controlled with
precision.
COM functions are now error trapped to prevent dialog boxes appearing in
server versions. Error messages are captured to the ErrorText property.
The Script Filter's default script can now be set.
Fixed display issue where memo field containing multi-line text (such as the
search and replace fields) scrolled to the bottom rather than displaying text
from the top.
The command line text of the External Program filter now does not have double
quotes removed from it.
TextPipe now can paste more than 64K of text at a time into a memo control.
Previously controls could hold more than 64K of text, you just couldn't paste
that amount.
TextPipe now forces text paste operations to be plain text.
Speed improvements with string translation caching.
Wizard form now aligns correctly.
A new option to only output modified files was added to the Output filter.
A new Search option on the File Menu allows filters to be easily located.
The status window now shows how many bytes of the input file have been
processed.
A quick short cut is now to double click on any of the filename fields to
simply open the file (e.g. Single file output).
The Remove and Remove All buttons now work for all tabs (previously they were
not used for the Comments, Scratch Pad and Trial Run tabs).
When filters are being run from the command line, a new blank filter window
is no longer created - this means that the status of the already running
filter can be easily seen.
The Execute Program filter now deletes the Input and Output files when
processing has completed. This filter is commonly used for sorting very large
data files and other external processing.
The context grep filters Remove matching/non-matching lines and Retain
matching/non-matching lines have been re-designed so that they work exactly
the same as the Select matching/non-matching lines filters.
The /I command line option (recurse subfolders) now also applies to file
lists loaded using /L=.
Added new command line option /CLEARFILES to clear the file list. Also added
/S0-/S6 to handle different search/replace types.
Fixed bug with Remove/Retain Lines Matching Perl Pattern List, where all
backslashes needed to be doubled twice.
Fixed bug with none of the /C... command line parameters working.
Fixed problem with Extract Email Addresses filter removing one extra
character from the start of the email address when it started with mailto:
sendto:, email= etc.
Improved window behaviour when TextPipe is activated from COM - the main
window is now not visible unless it is specifically shown.
Improved COM handling - TextPipe exits correctly now that there are no
internal references to the Application object.
Fixed problem with saving Count Duplicate Lines and Remove Duplicate Lines
filters.
Fixed problem with Remove Patterns From List filter - was crashing.
Fixed problem with multiple output filters overwriting the same file.
Fixed sub filters not working for filters Add Text Side by Side, Add
Repeating Text Side by Side, Remove Lines Found In, Retain Lines Found In,
Remove Matching Lines Found In, Retain Matching Lines Found In.
Added new options to the Export Command Line of the Output filter.
Fixed problem with memos not displaying text properly.
Added new Find in HTML helper window to help users Edit, Remove, Replace or
Append to text inside HTML tags (such as the ALT tag of IMAGES), between HTML
tags (such as between and ), and between two arbitrary pieces
of text.
Added new 'Select each line in turn' filter. This is a short cut for creating
a 'Select matching lines' filter with a pattern of '^'.
Window Menu now works properly - workaround found for Windows bug (being to
turn off AutoHotKeys and AutoLineReduction).
New System menu item (on task bar) to force TextPipe to close, even if it has
been activated via COM or from DataPipe etc.
Fixed double quotes appearing in filename fields after a filename has been
chosen.
When special characters are pasted, the default mode has been changed from
decimal to hex (so where \018 would previously have been pasted for special
character 18, now \x12 would be pasted)
New Word Wrap toggle item in the popup menu of memo fields.
New Start Column and Length options for Sort filter, Remove Duplicates filter
and Count Duplicates filter.
When a command line filter has been started with /G, TextPipe no longer
clears the filter list so that you can now see the contents of the command
line filter.
New command line parameter /OUTPUTKEEPSTRUCT to control whether the input
file's folder structure is retained when a new output folder is set.
Fixed a bug with dragging and dropping filters, where you could get an Access
Violation trying to edit the filter after moving it from inside its parent
filter.
Fixed @inputFilename and @fullInputFilename incorrectly using the output
filename.
Expanded the Special\Find and Replace menu item into an entire menu to make
it clearer.
Now uses Windows standard dialog boxes for easier translation.
Removed status window control resizing.
Removed validation on perl patterns that prevented back references from
working.
Fixed error with perl ^ not matching the start of the first line in files
less than 4K.
Fixed error with Extract Filter - some non-matching text was being passed to
the output.
Fixed the maximum allowed value on various filters - Split At Lines, Detail
lines allowed before writing to log file, size of random input file that can
be generated, insert column position, line numbering, context grep lines
before and after, delete columns values, duplicates filter, record length on
EOL filter, head/tail filter, all number-type filters like spaces to tabs,
centering etc, maximum buffer size for pattern matching etc.
Fixed handling of single and double quotes inside database queries.
Database filter 'insert script' option was not modifying file changed status.
Fixed TextPipe.getGlobalVar - now returns a blank string if the variable has
not been defined, instead of the variable name preceded by '@'.
GlobalVariables are now case-insensitive, and can hold multi-line values.
Fixed bug with 'Remove Prompting' not setting the modified flag if the filter
changed.
Changed the text qualifier for database insert scripts from " to '
All fonts are now set to the system default font. This makes translation of
Eastern languages far easier.
Tools\Put Sample In Trial Input now only includes files that have the 'Add'
action enabled. It also handles the situation where no files have been
specified.
Better DataPipe integration - now no longer hides the window when TextPipe is
closed. The main form minimizes instead.
Improved Command Line Wizard dialog.
New right click menu function for search/replace filters "Allow RTF tags"
converts the pattern to one that allows embedded RTF font tags.
Better integration with DataPipe - COM functions have changed, now
Application.hide minimizes TextPipe, and Application.show restores it.
Improved speed of COM functions.
Added COM code to support DataPipe
Fixed bug (again) where null lines caused the pattern matcher to fail.
Race condition fixed where a command line processing thread started with /G
/Q doesn't get a chance to start before the application exits.
New command line parameters /MAXIMIZED and /HIDDEN (hidden acts the same as
/MINIMIZED)
COM functions execute() and executeClipboard() now don't return until the
processing thread has completed.
New COM function isRunning to determine if a processing thread is still
running.
The COM function addReplaceFilter()'s ignoreCase parameter has been renamed
to matchCase - now the sense of the value is correct, and is consistent with
the user interface.
Fixed bug with perl-style patterns when used with look-behind assertions -
100 characters of text preceding the match is now retained in the search
buffer.
Secondary Output Filters can now be disabled and deleted.
Fixed errors occurring when a filter was modified after it had been moved up
or down using the up or down buttons.
Added Application.Quit function to COM
When created from COM, the default is now for the main form to be invisible.
Secondary Output Filters can now be added throughout the filter list to
duplicate the output file, or write selected lines and columns out to a
separate file, eg to place error lines or records in an error file, or to
separate records of different type or length into new files.
A new Select Block filter allows undelimited blocks of data to be selected
and processed as one.
Fixed VB/Jscript timeout - value is in milliseconds, not seconds.
Fixed obscure Perl replacement bug when run from command line.
Validation of ActiveX scripts now only occurs for VBScript - JScript
validation didn't work because the MS JScript engine does not implement
various functions.
Updated perl library to PCRE 3.7 (from 3.5)
EOL filter now allows more than one character in the replacement string.
Enabled some command line operations to provide Windows Explorer support in
TextPipe Web and Standard.
Fixed problem with the About dialog (in the evaluation version) interfering
with filter loading when TextPipe was invoked by double-clicking or opening a
.fll file.
Fixed bug with 'Escape Special Characters' not working properly.
Fixed bug with Split After XX Lines filter only outputting one line.
Fixed bug with 'Include Lines from End of File' when the number of lines was
greater than 255.
Tree view now displays more than just the first line of scripts and comments.
Fixed bug with perl-style grep and blank lines.
Input Filter export showed incorrect value for Delete Input Files After
Processing.
Filters loaded from the command line now have their full path saved, so they
get saved to the correct location even if the current directory is changed.
Added missing properties to Filter Export function.
Fixed bug handling command line filenames with embedded '..' paths.
New filter to Remove lines matching any patterns in a list.
New filter to Retain lines matching any patterns in a list.
New filters to Retain matching lines, Remove matching lines, Retain non-
matching lines and Remove non-matching lines - this is easier to understand
than the Select lines filters.
New filter to add a file side-by-side with an existing file.
New filter to add a file side-by-side with an existing file and repeat the
added file if the file being added to has more lines.
New filter to Split a File after a given number of lines.
New Input Filter to generate random a data file of a particular size.
New filter to repeat a file's contents a specified number of times.
Added Script Timeout option to VBScript/JScript filter
Show Command Line dialog now also shows the current path and all environment
variables.
Added 'Escape special characters' menu item to right-click menus of all
fields. This is very handy for adding backslashes in front of pattern
matching meta characters, and doubling '@' and '$' characters where
appropriate.
Added 'Allow Extra Spaces' menu item to right-click menu of all pattern
fields, allowing you to easily search and replace in HTML files by ignoring
any extra spaces or tabs that commonly occur.
Added 'Allow Extra Line Feeds or Spaces' menu item to right-click menu of all
pattern fields, allowing you to easily search and replace in HTML files by
ignoring any extra spaces, tabs or line feeds that commonly occur.
Updated help, fixed application links to help file, help file now displays
when TextPipe is first run
Fixed bug with filter loading
Made splitter in Search/Replace filter visible, now resizes correctly.
Fixed 'Add folder' button - newer versions of Windows caused this dialog to
appear behind the main window.
Unknown field codes (a name preceded by an '@' symbol) are now NOT replaced
with a blank - they are left as is to prevent inadvertent use of a field code
causing text to disappear. Known field codes (@clipboard, @inputFilename,
@fullInputFilename) still work as normal.
Added new COM function to filter windows - closeWithoutSave. Added new
options to COM interface, including default values.
Fixed bug with a filter validation function not being called when there were
sub filters anywhere above them in the filter list.
Fixed a bug affecting the Select Matching Lines, Select Non-matching Lines,
Select Lines From Length To END, Select Lines From Start To End, Select Lines
From Start For Length filters when immediately followed by a line-based
filter such as a Reverse Each Line, Pad To Width etc. Some text was not
output at all.
Fixed a bug affecting the Remove Lines Found In List and Only Keep Lines
Found In List filters - because the list was not internally sorted, the
actual lines would only occasionally be processed correctly. A workaround was
to ensure that the line list is first sorted in ascending alphabetical order.
Improved the email address extraction filter to automatically remove leading
'mailto:', 'mime:', 'smtp:', 'email=', and remove trailing '?Subject' and
trailing periods.
Fixed Perl Pattern Match not saving 'greedy' setting
Fixed hidden form problem with active field being invisible
Fixed Access Violations on editing filter list
PERL-STYLE pattern matching! - use both greedy and non-greedy matching
options, as well as embedded comments, assertions, non-capturing sub-
expressions and more!
Brief-style pattern matching
MS Word style pattern matching - use the *, ?, etc wildcards that you can use
in MS Word.
New 'Extract' option for search and replaces, all non-matching text is
discarded.
The Found Text can now be referred to in the Replacement Text as '$0' for
Exact, Sounds-like and Edit-distance matches. Previously the Found Text could
not be referred to, making some replacements harder.
You can now easily add a sub filter to the selected filter by holding [Alt]
down while choosing it from the menu or selecting its speed button.
You can now easily add a filter at the parent level of a sub filter by
holding [Shift] down while choosing it from the menu or selecting its speed
button.
New Input Filter setting for text to come from the Trial Run Input area.
The filters Add Lef - Margin, Add Righ - Margin, Add Header and Add Footer now
do not parse the included file for special functions or characters. This
ensures that the file being included is included exactly as is. The parsing
of text specified in the edit field continues as previously.
Fixed a recursion bug when [Alt+Down] is pressed to move a filter and its sub
filters down, now the filters are moved down correctly.
Fixed bug with base panel of search and replace filter disappearing after
returning from the Trial Run area.
Removed the Font Control context menu item for controls.
Better handling of Cancel button during a processing run.
Fixed handling of NULL values with database filter and Generate Insert Script
mode.
Database Command Timeout setting for queries that run for more than 30
seconds.
Improved Tree handling is quicker and easier to use; the treeview changes
immediately when a setting is changed, and there is a new Up and Down button
to easily move filters up and down (in addition to Alt+Up and Alt+down).
All command line defaults now come from the Normal Template normal.fll, not
the internal defaults which used to be:
inputFromFiles, promptOnEach=false, skip binary files,
binarySampleSize=100, outputToFiles, testMode=false, retainDateTime=false,
changeExtension=false, changeBaseDir=false.
Fix for the error " Th - Decision Cube Capacity is Low. Please deactivate
dimensions or change the data set"
Uninstall now removes all registry entries
Installer now supports unattended installs
Fixed short cut conflict between Remove/Remove All buttons and menu items
Fixed bug with registry security in registered version
Added 'Normal Template' functionality - a filter that is always loaded to set
defaults.
Delete Input Files after Processing setting now displays a warning box when
it is enabled. When a new filter is created this setting always defaults to
disabled.
Added shortcuts to tabs on status form, query replace form and command line
display form.
Fixed error in COM documentation for AddFile function - constants should be
1,2 and 3, not 0, 1 and 2.
Changed sizes of GUI controls to show foreign languages better.
Fixed bug with loading of foreign language, and enabled support for radio
group type controls.
Fixed bug with Append Mode.
Clicking on the filename column in the file list now sorts the filenames.
Improved COM interface - new methods for compiling the filter list which
improves speed if the same filter is used repeatedly
New Append to existing file setting
Localisation complete - now everything in TextPipe can be translated to a
foreign language by translating the file Languages\English.ini
For more details on how to translate TextPipe, please visit:
http://www.datamystic.com/textpipe/translate.html
COM interface now allows TextPipe to be fully controlled from languages such
as VisualBasic, VBScript, C++, Delphi etc
For more details on how to translate TextPipe, please visit:
http://www.datamystic.com/textpipe/com_interface.html
Fixed bug with dragging and dropping filters onto a filter that doesn't
support subfilters - now adds the dragged filter above it
Added code for TextPipe Pro Run Time Engine
- Marketing changes for evaluation edition
Began preparation for localisation
Fixed command line processing bug with modifying the Output filter settings
after loading a filter with the /F= command
Fixed bug with debug windows
Forms and menus can now be translated into other languages via a .ini file in
the Languages folder
Improved the load time of massive filter lists loaded via the /Z= command
line parameter
Trial Output now shows Column and Row in the same way that Trial Input does
File size for files larger than 2GB is now displayed correctly
Improved the way that form sizes and positions are saved
Improved the way that registered user data is saved under Win2K
Forced installation to require the Administrator account to prevent registry
writing problems under Win2K
Lines of context surrounding a matching line can now be displayed, both
before and after matching or non matching lines (grep filter)
Added short cuts to tabs (Alt+1 through Alt+5)
Added Filter Wizard to help new users
Added command line options for setting the output file extension, output
folder, and merge output filename.
Sub filters can now be created by holding down Shift as they are drag and
dropped to a location inside another filter
The database filter now directly supports creating a database insert script
from extracted data
Improved tutorial lessons
TextPipe tutorial now listed in the start menu
Fixed bug with default folder not being TextPipe's folder (default was My
Documents folder)
Fixed bug with access violation on exit after a filter is invoked from the
command line
Fixed bug with Count Duplicates filter missing some lines
Fixed bug with Extract Emails filter missing the last email address if it
ended at the end of the file
Fixed bug with errors not being reported during trial run - now if errors are
reported the status dialog will not automatically close
Changed default extension to .fll for link filter dialog. Also added an open
button so that the filter can be easily loaded and edited
Can now CHANGE THE REPLACEMENT TEXT during a run. This is very useful for
fixing up mistakes in the replacement text, and can also be used for pattern
matches
New Paste Special command on context menus to allow pasting of special
characters without TextPipe converting them to its internal code
Unknown '\' entries are now output with the leading '\' rather than without.
This is to help new users
Trial Input and Output no longer attempt to validate special characters such
as \t etc.
Fixed bug with '\' being incorrectly interpreted in the Prompt for replace
window
Fixed bug with filters becoming modified when Search/Replace filter was
selected (not edited)
VBScript/JScript code can now control TextPipe - LogError, LogInfo,
inputFilename, fullInputFilename, terminate, setGlobalVar, getGlobalVar,
subFilterEntireText, version
VBScript/JScript code can now have subfilters. Subfilters can be turned on or
off by the script at any time using InitSubFilters, FlushSubFilters,
subFilterChunk
Find/Find Again in all memo fields (multi-line edit)
Font control dialog on all edit fields (including memos)
Extended the number of filters than can be invoked via the command line
Facility to generate TextPipe command line parameters from an existing filter
list
Trial Run can now be interrupted - runs i - separate thread
Ctrl+A (Select All) now works in all fields
More options in the Window menu
Adhoc Transform tab renamed to Trial Run tab
Task bar menu now sends TextPipe to the back after a menu item is selected
Better handling of maximized windows when TextPipe restarts
Fixed bug with Select lines filter - if the input had (say) 2 lines and the
setting was from line 3 to END
Added drag and drop between filter windows
Added drag and drop promotion/demotion and movement of filters in the filter
tree display (and removed the Up/Down buttons)
Added Lesson 4 to tutorial
"x2" function now duplicates sub filters at the same time
Fixed bug with Insert Column At filter where the column was set to zero (ie
insert at end of line) and the actual line length = 1 (the text was inserted
at the beginning of the line)
Fixed bug with replace filter and whole words option - output text can be
duplicated
Fixed bug with merge filter and single file output filter
Fixed bug with Close All function not closing all windows
Fixed bug with Insert Filter File function changing the name of the currently
loaded filter
Updated Export To Clipboard and Print facility to cope with new filters and
tree structured filters. This facility is an excellent way to maintain
version information about your filters
Maps now moved t - separate 'maps' folder
textpipe.log is now created in the same folder as textpipe.exe if no path has
been specified
Recent Files menu now uses submenu of main menu
Filters opened will now load into the New Filter window if it is unchanged
and does not hold a filter
Fixed problem with files and folders added via the command line not
defaulting the subfolders setting correctly
Fixed problem with filter opening twice when opened from Explorer when
TextPipe isn't currently opened
Fixed bug with merge filter not performing any filters following it in the
filter list
Added ability for Windows Explorer to detect TextPipe and open a filter in an
existing TextPipe window
Recycle bin functionality temporarily removed while bugs fixed
Fixed buttons and help for Files Tab
Delete file operations now use the Recycle Bin
Fixed bug with Remove Lines From File and Only Show Lines From file filters
(Hopefully) fixed long-standing error on exit
New subscribe facility on help menu
Fixed regexp bug with characters sets [] involving ranges of characters
specified with a \ eg [\000-\255]. Also error message for character sets
containing a \000.
Better error handling when file delete/rename fails
Fixed bug with number of subfolders to process (1 Level was same as None)
File\Open now always opens a filter in a new window. New menu item to open in
the current window
Fixed bug with Remove button, only removed first file from file list even if
multiple files were selected
Now checks if log file exists before using Notepad to open it
Double quoting of filenames and folders containing spaces now handled
correctly
Better handling of dropped files
Added option to delete input files once they are processed - to support
integration with GFI FAXmaker
New Sorting options
ASCII Sort (case-sensitive and case-insensitive). Previously only an ANSI
sort was available (where lower case letters sort before upper case letters)
Numeric sort
Sort by length of line
New Unicode Map filter for translating Unicode files
Improved online help with Quick Start guide, improved tutorial and more
cross-referencing
Status form now has indicator to show whether test mode or overwrite mode is
active
All memo fields now can save to and load from a file
Trial output now has warning message if the output text contains null
characters
Improved Command Line Assistant dialog
Fixed bug with Query Replace window not scrolling found text into view
Fixed bug with "First Match Only" option discarding text
Fixed move filter up/down bug with filter lists containing more than 256
filters
Fixed bug with database filter not extracting data correctly when fixed width
format with no column titles was performed
Database filter now correctly handles non-SELECT queries that return a record
set
Fixed bug with access violation on exit
Fixed bug with pattern match ignoring First Match Only setting under some
circumstances
Added help toolbar for easier access to helpful features
Improved drag & drop behaviour for .fll files
Added automation check to disable debug filters
Added better error handling for MIM - decode quoted printable filter
Advanced logging details in results window
Added error checking to ensure file can be deleted/moved to correct final
filename, or error logged if not
Added output option for all files merged to a single file
New Clear All and Paste context menu item
Prompting dialog for Read Only files, Prompt Before Processing and Prompt on
Binary Files now has option for No To All and Yes To All
New Search/Replace option to only replace the first occurrence
Trial Input/Output now have horizontal scroll bars
Grep filters (Keep matching lines, remove matching lines) now as select
filters, where matching lines get subfilters applied to them
ActiveX scripting no longer requires dummy parameters for functions
Improved match information tab for Edit Distance and Sounds-like matches
Context menu on tree view for faster deletion/enable toggle
Database query now creates correct XML document. Handling of previously
unrecognised data types. Original query output as comment. Headers always
output even if 0 rows returned. Unique column names generated for all output
types. Encoding="UTF-8" removed from XML header - not supported by some
versions of IE
Status dialog now shows count of Non-binary Files Skipped
New Edit Distance option for search/replace filters - very handy for finding
and locating misspelt words
New option to retain the existing file list when opening a new filter
New facility for search/replace to replace all or none within the current
selection
Loading a .fll filter file now opens in the same window
Better detection of filter list changed
Better handling of loading very old filter lists
Added force lines to width filter
Added advanced logging facilities with detailed tracking of all changes made
Fixed exception with exact match filter
Fixed handling of non-select statements in Database filter, and improved
error checking code
Fixed directory where preliminary output file created
Fixed split filter - was not performing "start file" processing for following
filters
Fixed Remove Email Headers - was crashing
Fixed bug that affects all line-processing filters - when using Mac files the
first character on each lin - may be repeated
Fixed bug with exact match filter that caused text to be output multiple
times
Fixed Win NT bug with popup context menus on memo fields
Fixed delete columns filter
Fixed handling of grep/inverse grep filters
Comment filters now have vertical scroll bars for comment text
Fixed subfilter bugs where filter list was not built correctly, missing
several filters
Fixed bug with Select Lines filter not outputting its results
Fixed bug with Select CSV Fields filter
Fixed errors with start and end word for Edit Distance and Sounds-like
filters
Fixed split filter
Fixed handling of Select Columns with the 4th option
Fixed debug filter error
Fixed Select CSV/Tab field missing last field
Fixed handling of map menu and task bar menu
** Sub filters - drag and drop a filter onto another in the tree view and the
dropped filter will then process JUST the output from the parent node. Eg
drop a "Title Case" filter onto a "Search/Replace" filter to change the
search result to Title Case
** New Database Connection filter. Attach to any ADO/ODBC database and
execute any SQL command text against it. The result can be output as CSV,
Tab-delimited or XML (with automatically generated DTD schema). All
characters are appropriately escaped, including quotes within strings for
CSV, and XML special characters.
Pattern matching now extended to handle up to 36 ($0-9,a-z) bracketed
expressions. This is 26 more than any other product on th - market.
New Select CSV Fields filter - select a group of consecutive CSV fields and
apply sub filters to the result by dragging and dropping them onto this
filter. Handles commas embedded in quotes etc
New Select Tab-delimited Fields filter - select a group of consecutive tab-
delimited fields and apply sub filters to the result by dragging and dropping
them onto this filter
New Select Line Range filter - select a group of consecutive lines and apply
sub filters to the result by dragging and dropping them onto this filter
New Select Column Range filter - select a group of consecutive columns and
apply sub filters to the result by dragging and dropping them onto this
filter
New Remove All filter - useful for subfilters
Convert EOL filter can now specify any string as the new EOL character
New Truncate to Width filter
File output has new "Maintain folder structure" option when outputting to a
new folder
Remove Columns filter now has more advanced options
New option Skip Prompt if Identical for Search/replace filters
ActiveX scripts can now easily output text at the start and end of each file
New tools added - option to verify a filter before it is run - performs
syntax check etc. Also added Windows calculator
Explorer extension - right click on files, folders or drives brings up
TextPipe options. New window on Tools menu to customise the Explorer
extensions
Command Line Assistant window - provides examples of how to run TextPipe from
the command line in a variety of modes
Show Command Line window - displays the command line parameters used to
invoke TextPipe - useful for debugging command line scripts
Check for TextPipe Updates window - check if you have the most recent
TextPipe by querying the DataMystic web site
TextPipe now interfaces to 2 external tools (see Help\Related Products menu)-
JustData from www.justsoft.com.au
Offline Explorer from www.metaproducts.com
GUI enhancements
Tree view for filters which allows multi-level filters to be collapsed
Multiple-document interface (MDI) allows multiple filters to be open at once.
New Window menu to support MDI interface
Dockable toolbars
Icons in menus
Right-click (context) menus on all edit/memo fields allow special characters,
common patterns, replace expressions and macros to be inserted, as well as
the normal clipboard operations
Can now set the font used in the Ad-hoc Input/Output area
Bug fixes
ActiveX script now works in non-ad-hoc mode
Trial Run area now does not interpret \n, \t etc
Split files at Character - fixed bug
Example code for the TextPipe Engine - a text transformation .dll that can be
downloaded from www.datamystic.com
New ActiveX scripting filter supports arbitrary user-written code in
VBScript, Jscript or PerlScript, PythonScript, REXXscript etc
Output To New Folder can now recreate multi-level folder trees and output to
a new drive
New Undo command to undo any changes to the current filter
New file grid button to Copy Row
New Extract filter to extract just the required text
Show Duplicates filter now called Count Duplicates, and outputs the number of
times found to the left of each item
New Tools menu item to pull sample data from the file list or clipboard into
the trial run area
New Tools menu item to move the Trial Run Output back to the Trial Run Input
Add Line Numbers filter now allows format of line numbers can be changed to
custom user-defined settings, including numbering in hexadecimal etc
Sort filter now has Ignore Case option
Header, footer, lef - margin or righ - margin can now insert either a file or
text. Cursor position now shown
Length of trial input and output area shown in both lines and characters.
Cursor position now shown
Warning indicator on status bar now shows if Test Run mode is active or not
Debug filter - creates a debug window showing the text passing through the
filter at that point
External filter files can now be linked to, providing true 'black box' style
re-use, using File\Link To Filter
Can now duplicate the current filter - this makes adding a large number of
similar filters very easy
Find and replace filters now use multi-line edit control - making it easier
to edit large selections. The new edit control also provides the length and
line count of the text
Maps now display a hex column in addition to the origina - Decimal and ASCII
columns
Changed handling of changing the replacement text for a pattern-based search
- now the default text is not modified for future replaces
Tips of the Day
Query replace window now supports resizing better
Changed registration URLs
The filter names shown in the filter list are now truncated, preventing
Windows crashes with very long filter names
Fixed logging functions which were going wild when more than 1000 files were
processed
Fixed pattern matching not handling \n \r \e \t \a
Fixed Randomize filter - was outputting blank text
Fixed pattern based search with patterns longer than 247 characters - was
truncating and matching early
Soundex search method to search for words that sound like another word.
Useful when you are replacing a typing error
New Preferences dialog
- Play sound on completion
- If no association, use user defined editor (need new options window)
Prepare filter for automation - removes all user prompting
Prepare filter for distribution - removes all test text
Can now insert a file with the Add Header, Add Footer, Add Lef - Margin and
Add Righ - Margin filters - browse button
Add Line Numbers now has option to skip blank lines, (ie omit count but keep
counting), as well as omit count and not count line)
Now can split files at a given character, after a specified number of
occurrences (and split before, after or on top of character)
ANSI to Unicode and Unicode to ANSI filters (single byte to double byte)
Comment filters now shown in italics, disabled filters shown with strike
through text, speed button to add comments
Better progress indicator - now much more accurate
Duration of filter job in seconds shown in status dialog
Increased size of edit fields for easier editing of multi-line text. Display
of text length and number of lines of all memo fields (like Header, Footer,
Left/Righ - Margin, Insert Column etc)
Performance improvement when handling very large files
Improved performance for Remove Blanks from Start of Line, Convert to Lower
Case, Convert to Upper Case, OEM to ANSI, ANSI to OEM
Query replace dialog now shows file currently being processed
Query replace dialog now shows filter information for both normal and pattern
based searches (so it is easy to see which text was matched by each
expression)
Improved command line automation support. Now can have comments embedded -
useful when converting Perl scripts to TextPipe scripts because code can be
commented out rather than edited out
Form resizing now handled better
Fixed bug with Query Replace when replacement text was modified before
replacing
Fixed bug with saving/loading of window positions
Fixed bug with detection of when the filter has been modified
Fixed bug with query replace 'Yes - file' button - did not change the first
entry
Name change from DropConvert to TextPipe
Regular expression search and replace - allows pattern matching (also known
as non-exact matches or regexps). Case-insensitive or case-sensitive matching
HTML Help integration
Prompt on replace. Can change the replacement text
Option to delete test files at end of test run
@inputFilename macro now allowed in filters for Add Text To Start Of Lines,
Add Text At End Of Lines, Add Column Of Text
New @clipboard macro to insert the clipboard contents
Can disable filters by double-clicking or pressing Ctrl+T
When either the error display or the file display is full it is sent to the
log file
/z command line parameter to read parameters from a file
Grep-based regular expression matching
In-place editing of file specifications
Inclusion and exclusion lists for file processing
File groups allow better handling of complex file groupings
Subfolder recursion can now be specified on a per-file basis. Improved
performance of subfolder recursion
Improved detection for binary files - sample size can be adjusted, and binary
characteristics are better identified
Case-sensitive replace - if found text is uppercase, replacement can be
forced to all uppercase etc
New Export Filter to clipboard feature. Useful for documenting filters in
Word
Enhanced detail when printing filters
File associations for .dCv test files
Status dialog now show - separate error tab, and shows files generated by each
filter, with the old and new size of each with a comparison display for quick
checks
Install/uninstall feature. Creates Start Menu items, desktop icon, Send To
menu and file associations for .fll and .map files and for text files
Scratch pad area for test text
Trial run area for iterative testing
Improved error log for command line initiation
Pasted text is scanned for binary characters - and they are converted to
readable text. Pasted text is now checked to see if it has already been
converted
Heaps of useful sample filters
Window sizes now saved and restored. Minimum window sizes enforced
File list now supports including and excluding files, file groups, subfolder
inclusion/exclusion on a per-line basis
Scratch Pad area for saving chunks of text
New Edit Menu with Cut, Copy, Paste and Disable/Enable Filter
Settings now saved on a per-user basis
Improved descriptions in filter window
Improved performance. Better use of memory (now uses far less)
Scrolling on drag of filters in filter list
More filters and options available from command line
Now handles filters from future versions better
Printer setup menu item now doesn't get lost :-)
Fixed bug with "Base map on filter list" function
Filter to insert a column of text
Filter to convert to ranDOm CasE
Filter to extract URLs. Very useful for retrieving mailto, http, https, ftp
and gopher addresses
Task bar menu. Allows filters to be loaded and run without leaving external
programs such as Word, enabling conversion to occur seamlessly without
leaving the currently running program
All status windows from completed tasks are closed when starting a new task
Input filter new options:
Input from file or clipboard
Output filter new options:
Output to file or clipboard (independent of Input filter)
change file extension
change base directory on output
Filter to only show duplicate lines
Filter to reverse each line
Filter to centre justify lines
Filter to right justify lines
Filter to remove lines contained in an external file
Filter to only show lines contained in an external file
Filter to run external program. Very useful for user-written filters or for
features not yet incorporated into DropConvert
Filter to add comments to the filter list
Character maths filter now shows hex and binary equivalents for operand
File history (up to 5 filters) shown in file menu
Tool bar buttons for commonly-used filters
Floating hints now used in preference to status bar
Explorer menus for .fll file type
Removed evaluation edition restriction of 5 filters
Now uses InstallShield install/uninstall
See http://www.datamystic.com for more details on TextPipe
As always, TextPipe will read filters from ALL old versions without any changes being required.
New Features
What's New in TextPipe v12.1 - August 14, 2024
Registered Users / Current Customers
Login and download the upgrade, and/or see if you are
entitled to an upgrade
Related Applications You May Need
If you're a consultant and you don't want your clients to edit your filter, use TextPipe's No Editing Edition.
If you're an application programmer and want to embed TextPipe in your application, see the TextPipe Engine.
See also:
WordPipe - search
and replace Microsoft Word documents
ExcelPipe - search and replace Microsoft
Excel spreadsheets
PowerPointPipe - search and replace
Microsoft PowerPoint presentations
Detailed Changes Log
What's New in TextPipe v12 - June 6, 2022