Scanning a line for a code - then using that code to update the related line.

Get help with installation and running here.

Moderators: DataMystic Support, Moderators

Post Reply
PGB20201
Posts: 1
Joined: Tue Mar 09, 2021 12:02 am

Scanning a line for a code - then using that code to update the related line.

Post by PGB20201 » Tue Mar 09, 2021 12:24 am

Hi All,

I'm currently evaluating Textpipe Pro to see if it is able to solve an issue I currently have.

I have some text files with two distinct sections, each section has a set of lines of text which are related to each other via an 'entry number'. I need to take an extract of text from one of the lines from the first section and append it to the end of the corresponding line (with the matching 'entry number') in the second section.

Here's a simplified example:
U-005X;GLASGOW INTL;LIMO,CF-LIMOREF1/VNO6000/PICK UP TIME:0900;
U-002X;GLASGOW INTL;FERRY,CF-FERRYREF1/VNO7733/DEPARTURE TIME:1100;
U-003X;GLASGOW INTL;TAXI,CF-TAXIREF1/VNO7418/PICK UP TIME:1000;

RM*MS/PC5/SG02/PALL
RM*MS/PC11/SG03
RM*MS/PC11/SG05

In the above example, I need the the pick up/departure time in the first section appended to the corresponding line in the second section. The two sections are linked via the 'entry number' which is the bold underlined number above.

So far I've managed to work out the following:
  • Extracted the Entry Number into @SegNum using Regex.
  • Extracted the Time into @Time using regex.
What I was planning on doing was having another Find Pattern filter which then used @SegNum in the regex pattern, however I can't figure out how to:
  • Cut @SegNum down from three characters to two characters to match the second section.
  • Use @SegNum in a regex match.
Is what I'm trying to do possible, and if so, would someone tell me where I'm going wrong? When I write @SegNum into the regex pattern section it doesn't seem to be resolving to the variable value as far as I can tell.

User avatar
DataMystic Support
Site Admin
Posts: 2237
Joined: Mon Jun 30, 2003 12:32 pm
Location: Melbourne, Australia
Contact:

Re: Scanning a line for a code - then using that code to update the related line.

Post by DataMystic Support » Thu Apr 01, 2021 7:35 am

Hi there,

TextPipe isn't designed to solve the problem this way.

In principle, consider that TextPipe processes your entire file with the first filter, before passing it into the next filter. So the cause-effect you're after here will only work for the last occurrence. Also, regex matches are built at the start of the job for speed - they're not self-modifying.

The best way to approach this is to
1. Add the 'entry number' to the start of each line, for both types of line
2. Sort the file, so that similar entry numbers end up next to each other
3. Use a 2-line search/replace to shift the data around
4. If necessary, remove the entry number and unwanted lines using a marker character.

This approach is easier for TextPipe to parallelize and will work with files of ridiculous size.

Hope this helps!
Regards,

Simon Carter, https://www.DataMystic.com
https://www.JadeDiabetes.com - Insulin dose calculator for Type 1 diabetes
https://www.DownloadPipe.com - 250,000 free software downloads

DFH
Posts: 975
Joined: Sun Dec 09, 2007 2:49 am
Location: UK

Re: Scanning a line for a code - then using that code to update the related line.

Post by DFH » Thu Apr 08, 2021 7:04 am

btw. TextPipe Standard would suffice for this task.

TextPipe Pro is only really an advantage if you're processing data from mainframe computers.

I've been using TextPipe Standard for almost 20 years.
It's the best thing since sliced bread.
David

Post Reply