Text parsing for large files

Login to reply to this topic.
Thu, 2005-11-17 10:17
Joined: 2005-10-10
Forum posts: 37
Hi...

I'm back with a doubt in text parsing... I have a large file to parse which contains about 52 lines of text.

THe constraints of this issue is that:

1. The size of the file is 1.5KB.
2. The delimiter in the file is : ": " (Colon and a space)
3. TBuf is not accepting size greater than 1184.

Is there a way of reading the text file containing the data into a HBufC and later converting it into corresponding TBuf as the TLex class takes only TDesC, TBuf as input.  How do i go about this???

The Cruise is on...


Thu, 2005-11-17 13:00
Joined: 2003-10-10
Forum posts: 123
Re: Text parsing for large files
all depends on the format. failing that you can read one byte at a time
Thu, 2005-11-17 13:22
Joined: 2005-10-10
Forum posts: 37
Re: Text parsing for large files
I'm attaching the text file tat has to be parsed...

Check this out and please temme:
Nov  7, 2005 (Close)
BSE Sensex : 8206.83 (+1.66%)
A.C.C.: 462.15 (-0.60%)
BAJAJ AUTO: 1767.25 (+0.71%)
BHARTI TELE: 337.50 (+2.55%)
BHEL: 1191.10 (+4.65%)
CIPLA: 375.05 (+4.35%)
DR.REDDY: 888.05 (+4.90%)
GRASIM: 1219.15 (+3.41%)
GUJ.AMBUJA: 73.00 (-0.14%)
HDFC: 1005.30 (+0.35%)
HDFC BANK: 624.65 (+0.73%)
HERO HONDA: 749.45 (+4.91%)
HINDALCO: 124.70 (+3.31%)
HLL: 167.25 (+1.49%)
ICICI BANK: 499.50 (+0.09%)
INFOSYS: 2637.10 (+1.87%)
ITC: 125.80 (+0.64%)
L&T: 1514.60 (+0.34%)
MARUTI: 562.20 (+3.72%)
NTPC: 98.15 (+0.31%)
ONGC: 945.55 (+1.49%)
RANBAXY: 371.70 (+5.70%)
REL. ENERGY: 538.55 (+2.09%)
RELIANCE: 787.10 (+1.29%)
SATYAM: 635.55 (+1.27%)
SBI: 834.75 (+0.51%)
TATA MOTORS: 493.25 (+2.62%)
TATA POWER: 428.50 (+3.95%)
TATA STEEL: 363.90 (+4.66%)
TCS LTD.: 1458.65 (+2.33%)
WIPRO: 377.20 (+2.00%)

Thanx in advance...

The Cruise is on...

Thu, 2005-11-17 13:27
Joined: 2005-10-10
Forum posts: 37
Re: Text parsing for large files
This file has to be split as follows:

A.C.C and 462.15 (-0.60%).

I want this parsing to be done for a specified number of lines, say 10 or 15 only and discard the rest of the file...

How do i go about this???

The Cruise is on...

Thu, 2005-11-17 14:07
Joined: 2005-09-26
Forum posts: 36
Re: Text parsing for large files
I don't have much related experience, but I would look into using RFileBuf or RStreamBuf objects instead of HBuf, I think those may not have a maximum size or may have a larger max size, since they are specifically for parsing files.
Fri, 2005-11-18 03:47
Joined: 2005-10-10
Forum posts: 37
Re: Text parsing for large files
Will check up those 2...

Thanx...

The Cruise is on...

Fri, 2005-11-18 06:31
Joined: 2005-03-15
Forum posts: 94
Re: Text parsing for large files
You could try to use CLineReader for reading a line at a time and do the parsing at the same time.

Here is the sample code:

Code:
RFs fsSession;
User::LeaveIfError(fsSession.Connect());
RFileReadStream readStream;
readStream.PushL();

User::LeaveIfError(readStream.Open(fsSession, KFile, EFileRead));

CLineReader* myReader = CLineReader::NewL(readStream);
TInt aPos = 0;
TInt aErr = 0;
while(aErr!=KErrEof)
{
myReader->ReadLineL(aPos, aErr);
// Do your parsing here ..
}

CleanupStack::PopAndDestroy();

Vivek

Vivek Chopra

Fri, 2005-11-18 08:19
Joined: 2003-10-15
Forum posts: 78
Re: Text parsing for large files
Quote from: ashwinmt
3. TBuf is not accepting size greater than 1184.

Minor comment: Symbian stack size is quite limited and call stack uses the same space, that variables stack. If you allocate A LOT of things on the stack at some point your program might start mysteriously fail again and again. Just when you try to call some function, you always used without problem.
These stack overflow bugs are rather difficult to debug. So as a rule of thumb allocate everything you can on the heap.

Of course, it doesn't mean that you HAVE to do it. It is ok to allocate ints, handles or small structs on the stack, but kilobyte-sized things should be on the heap.

Sat, 2005-11-19 11:01
Joined: 2003-10-10
Forum posts: 123
Re: Text parsing for large files
Ideally you should push the myReader variable incase it leaves.
Wed, 2007-03-21 18:48
Joined: 2007-03-21
Forum posts: 2
Re: Text parsing for large files
Hi,

I am trying to use the given code example to read a text file and my application is crashing at:

CLineReader* myReader = CLineReader::NewL(readStream);

can you please help?

Thanks in advance,

Regards,
Angela
Wed, 2007-03-21 19:22
Joined: 2004-07-10
Forum posts: 364
Re: Text parsing for large files
Quote from: Cruisemaniac
Hi...

I'm back with a doubt in text parsing... I have a large file to parse which contains about 52 lines of text.

THe constraints of this issue is that:

1. The size of the file is 1.5KB.
2. The delimiter in the file is : ": " (Colon and a space)
3. TBuf is not accepting size greater than 1184.

Is there a way of reading the text file containing the data into a HBufC and later converting it into corresponding TBuf as the TLex class takes only TDesC, TBuf as input.  How do i go about this???


Look at the descriptor class hierarcy, a TBuf and a HBufC are both TDesCs.

Wed, 2007-03-21 20:26
Joined: 2006-02-17
Forum posts: 7
Re: Text parsing for large files
Hi,
You can also use TFileText to read on line at a time..

Best of luck..
Thu, 2007-03-22 16:12
Joined: 2006-10-07
Forum posts: 131
Re: Text parsing for large files
Quote from: mungbeans
Look at the descriptor class hierarcy, a TBuf and a HBufC are both TDesCs.
Note the post date: the dude posted 1.5 *years* ago. I am willing to bet he has already figured it by now. If he didn't, no amount of advice would help. Cheezy
Thu, 2007-03-22 17:50
Joined: 2004-07-10
Forum posts: 364
Re: Text parsing for large files
Nah, I bet after 1.5 years of trying to use descriptors he still hasn't got them totally figured out.
  • Login to reply to this topic.