File, Streams and Stores

Login to reply to this topic.
Fri, 2005-06-24 08:43
NewLC AdministratorSymbian AccreditedForum Nokia Champion
Joined: 2003-01-14
Forum posts: 1918
A few definitions regarding files, streams and store (taken from this post by Didster):


Files:  Just plain old binary files, where you the developer are responsable for writting everything to that file in the correct format that you need to get data back out.  For instance, if you write a descriptor to the file, you would need to write a flag indicating its type (8 or 16 bit), it's length, its maxlength if its non-constant, and its data to reconstruct it from the file.

Code:
   /* Txt file to be opened. */
   RFile file ;
   file.Open(aFs, aFilename,EFileRead) ;
   CleanupClosePushL(file);

   /* Size of file. */
   TInt iSize ;   
   file.Size(iSize) ;
   
   /* Allocate enough memory to read in the whole file. */
   HBufC8* buf = HBufC8::NewL(iSize) ;

   /* Get a pointer to modify contents of buffer. */
   TPtr8  pBuf = buf->Des() ;

   /* Read data into buf. */
   file.Read(pBuf) ;

   CleanupStack::PopAndDestroy() ; // file






Streams:  Provide a simple API for you to write data in external format and to allow you to abstract the underlying storage away from your code.  For instance, you can attach a stream to a file, or to a block of memory, but reading and writting to it remain the same.  Also, the bit above about writting lengths etc is done for you - when you send a descriptor to a stream, everything needed to reconstruct it is written for you.  This is because the descriptor classes serialise themselfs - they provide an Externalise and Internalise method which the framework calls when one of them is added to a stream - in these method they write and read all their own data members.  This process is known as serialisation, ever done MFC?  Then think CArchive - basically its about making it a classes own responsabilty to save and load thier state.

Code:
//
// Open a RFs
RFs aFs;
User::LeaveIfError(aFs.Connect());
CleanupClosePushL(aFs);

_LIT(KStreamName, "C:\\TestStream.dat");

//
// Delete old file
aFs.Delete(KStreamName);

//
// Open a file write stream
RFileWriteStream fWrite;
User::LeaveIfError(fWrite.Create(aFs, KStreamName, EFileWrite));
CleanupClosePushL(fWrite);

//
// Write to the stream and commit it
_LIT(KTextToCopy, "This is a test stream");
fWrite << KTextToCopy(); // Warning, this _CAN_ leave
fWrite.CommitL();

//
// Close the stream
CleanupStack::PopAndDestroy(&fWrite);

//
// Open a read stream
RFileReadStream fRead;
User::LeaveIfError(fRead.Open(aFs, KStreamName, EFileRead));
CleanupClosePushL(fRead);

//
// Read data back from the stream.  Using the HBufC::NewL which takes a RReadStream
// so we get a buffer exactly the right size for whats in the stream
HBufC* pBuf = HBufC::NewL(fRead, 1000);

//
// Close the read stream
CleanupStack::PopAndDestroy(&fRead);

//
// Close the RFs
CleanupStack::PopAndDestroy(&aFs);





Stores:  Streams are great, but they have one problem.  They are sequentially accessed.  In that you must know the exact order thinks went in to get them out.  So say if you have two classes in your stream, A and B.  When you load from that stream, if you want B, you must first load A even if you don't want it.  Stores get round this.  Essentially, they are a network of streams - mutiple streams which are individually accessable from there "stream id".  The stream id's are maintained by a dictonary at the start of the store.

Code:
//
// A class for correct cleanup of RPointerArrays
template <class T>
class CleanupResetAndDestroyClose
{
public:
inline static void PushL(T& aRef)
{CleanupStack::PushL(TCleanupItem(&Close,&aRef));}

static void Close(TAny *aPtr)
{
(STATIC_CAST(T*,aPtr))->ResetAndDestroy();
(STATIC_CAST(T*,aPtr))->Close();
}
};

template <class T>
inline void CleanupResetAndDestroyClosePushL(T& aRef)
{CleanupResetAndDestroyClose<T>::PushL(aRef);}

//
// A class we wish to put into the store
class CFoo : public CBase
{
public:
CFoo() {}
TBuf<20> iName;
TInt iSomeData;

TStreamId StoreL(CStreamStore& aStore) const;
void ExternalizeL(RWriteStream& aStream) const;
void RestoreL(CStreamStore& aStore,TStreamId anId);
void InternalizeL(RReadStream& aStream);
};

//
// Create a stream, and externilize ourselfs into it
TStreamId CFoo::StoreL(CStreamStore& aStore) const
{
RStoreWriteStream outstream;
TStreamId id = outstream.CreateLC(aStore);
ExternalizeL(outstream);
outstream.CommitL();
CleanupStack::PopAndDestroy(&outstream);
return id;
}

void CFoo::ExternalizeL(RWriteStream& aStream) const
{
aStream << iName;
aStream.WriteInt16L(iSomeData);
}

//
// Open up the given stream, and restore from it
void CFoo::RestoreL(CStreamStore& aStore,TStreamId anId)
{
RStoreReadStream stream;
stream.OpenLC(aStore,anId);
InternalizeL(stream);
CleanupStack::PopAndDestroy(&stream);
}

void CFoo::InternalizeL(RReadStream& aStream)
{
aStream >> iName;
iSomeData = aStream.ReadInt16L();
}

//
// Add each record to the store
void DoSaveToStoreL(CFileStore* aStore)
{
RPointerArray<CFoo> rObs;
CleanupResetAndDestroyClosePushL(rObs);

CFoo* pFoo1 = new CFoo;
CleanupStack::PushL(pFoo1);
pFoo1->iName.Copy(_L("Test Ob1"));
pFoo1->iSomeData = 1;
User::LeaveIfError(rObs.Append(pFoo1));
CleanupStack::Pop(pFoo1);

CFoo* pFoo2 = new CFoo;
CleanupStack::PushL(pFoo2);
pFoo2->iName.Copy(_L("Test Ob2"));
pFoo2->iSomeData = 2;
User::LeaveIfError(rObs.Append(pFoo2));
CleanupStack::Pop(pFoo2);

RArray < TStreamId > rIndexArray;
CleanupClosePushL(rIndexArray);

//
// Store each object from rObs into the store.  Each object will create it's own stream
// in the store, and it's id will be returned from ->StoreL.  We then store the ids in a
// index which we store in the root stream of the store.
for(TInt n = 0; n < rObs.Count(); ++n)
{
TStreamId id = rObs[n]->StoreL(*aStore);
User::LeaveIfError(rIndexArray.Append(id));
}

//
// Write index
RStoreWriteStream rs;
rs.ReplaceLC(*aStore, aStore->Root());

rs.WriteInt16L(rIndexArray.Count());

for(TInt nn = 0; nn < rIndexArray.Count(); ++nn)
{
//
// Write name
rs << rObs[nn]->iName;

//
// And index
rs << rIndexArray[nn];
}

rs.CommitL();
CleanupStack::PopAndDestroy(&rs);

//
// Commit the store
aStore->CommitL();

//
// Done with the index array
CleanupStack::PopAndDestroy(&rIndexArray);

//
// And the objects
CleanupStack::PopAndDestroy(&rObs);
}

LOCAL_C void doExampleL()
{
//
// Open a RFs
RFs aFs;
User::LeaveIfError(aFs.Connect());
CleanupClosePushL(aFs);

_LIT(KStoreName, "C:\\TestStore.dat");

//
// Create a perminate file store
CPermanentFileStore* pStore = CPermanentFileStore::ReplaceLC(aFs, KStoreName, EFileWrite);
pStore->SetTypeL(KPermanentFileStoreLayoutUid);

//
// Write a fake root stream containg 0 for the entry count
RStoreWriteStream rs;
TStreamId id = rs.CreateLC(*pStore);
rs.WriteInt16L(0);
rs.CommitL();
pStore->SetRootL(id);
pStore->CommitL();
CleanupStack::PopAndDestroy(&rs);

//
// Generate some objects and save them to the store
TRAPD(err, DoSaveToStoreL(pStore));
if(err != KErrNone)
{
pStore->Revert();
User::Leave(err);
}

//
// The store now contains a series of streams, and each stream contains it's own CFoo, which
// we can load independatly of each other

//
// Close the store
CleanupStack::PopAndDestroy(pStore);

//
// Open up the store for reading
CPermanentFileStore* pReadStore = CPermanentFileStore::OpenLC(aFs,
KStoreName, EFileRead);

//
// Go through the ids, and look for the one called Test Ob2 to demonstrate
// we don't need to load Test Ob1 to get at Test Ob2 as we would if we were
// using a normal stream
TInt nRecordCount = 0;

RStoreReadStream streamIn;
streamIn.OpenLC(*pReadStore, pReadStore->Root());
nRecordCount = streamIn.ReadInt16L();

TStreamId nTheId;

for(TInt n = 0; n < nRecordCount; ++n)
{
//
// Read name
TBuf<20> iName;
streamIn >> iName;

TStreamId id;
streamIn >> id;
if(iName.Compare(_L("Test Ob2")) == 0)
{
//
// Found it!
nTheId = id;
break;
}
}

CleanupStack::PopAndDestroy(&streamIn);

if(nTheId != KNullStreamId)
{
//
// Load er back in
CFoo* pOb2 = new CFoo;
CleanupStack::PushL(pOb2);

//
// Restore it
pOb2->RestoreL(*pReadStore, nTheId);

//
// Do something with it
// .
// ,
//
// Destory it
CleanupStack::PopAndDestroy(pOb2);
}

//
// Close the store
CleanupStack::PopAndDestroy(pReadStore);


//
// Close the RFs
CleanupStack::PopAndDestroy(&aFs);
}

Eric Bustarret
NewLC Founder & CEO / Professional Symbian OS Consultant


Fri, 2005-06-24 09:05
Joined: 2005-06-13
Forum posts: 68
Re: File, Streams and Stores
Cheers to eric and didster for the insight!

Well done heroes  Smiley,

Nikolas.

If we fall down it's so we can learn to pick ourselves up.

Tue, 2005-07-26 08:33
Joined: 2005-04-20
Forum posts: 9
Re: File, Streams and Stores
Thanks。
It is very helpful for me.

qiangsir.

I love symbian.

Wed, 2005-07-27 03:56
Joined: 2005-03-03
Forum posts: 56
Re: File, Streams and Stores
It has been observer that, even if i update the same entry in a store, same no of bytes but different letters/chars the file size increases. why?
regards
arun
Sat, 2005-07-30 15:24
Joined: 2004-07-28
Forum posts: 1379
Re: File, Streams and Stores
It's all down to how the stores are implmented, and nothing to worry about.  Eventually, the store will compact itself, and the wasted space will be freed.  If you wish to force it, just call ->CompactL followed by ->CommitL on the store, and it will shrink down to its real size.

didster

Mon, 2005-08-01 13:16
Joined: 2005-03-16
Forum posts: 1
Re: File, Streams and Stores
Hello

I have a piece of code:

RStoreWriteStream writeStream;
CleanupClosePushL(writeStream);
RFs fs;
User::LeaveIfError( fs.Connect() );
_LIT( KStreamStoreName, "C:\\Test.txt" );

CFileStore* store = CPermanentFileStore::ReplaceLC( fs, KStreamStoreName, EFileShareAny );
TStreamId id = writeStream.CreateLC( *store );

This code works fine with my friend but I get System error -18 from the CreateLC line. I have reinstaled my S80 SDK and the same problem is also with S60 2nd Ed. I also check that my hard disk is not full. Anyone can explain? Thank you.

Tue, 2005-08-09 13:49
Joined: 2005-08-09
Forum posts: 29
Re: File, Streams and Stores
Hi,
Is it posible to do something like File Mapping (CreateFileMapping(...) ) in Win32 , to get a pointer to the raw data in memory (file).

Thank you.

Vladan Markovic
Fri, 2005-08-26 09:40
Joined: 2005-08-26
Forum posts: 38
Re: File, Streams and Stores
Well  in one file I have used RFile( RFs) to open a  dict.cfg file and i loaded the file into an array but after closing tht file  i started a Agenda server  to open the calendar file but my application could load the  dict.cfg using rfile but  when it reaches at the point of code where Agenda server starts it returns me with an error (-15)  . but this code is working perfectly alright in  another pc wht could be the reason. it also shows agnmodel.dll  access violation wht it coul be ? 
Thu, 2005-09-22 13:49
Joined: 2005-09-14
Forum posts: 3
Re: File, Streams and Stores
Hi,

I have compressed video file in my PC. I have to send 1kb of data 
from  file to the blutooth dvice ( Nokia6630) memroy.
For that any symbian API is there or  other wise how can I write a piece code for that ,
Can you please suggest me.

Thanks.
Sat, 2005-09-24 15:09
Joined: 2005-09-24
Forum posts: 3
Re: File, Streams and Stores
Hi! I tried the code for streams and it worked fine for a 1KB .dat file.
I tried a 20KB .jpg file and it didn't work.
Can somebody help me?
Mon, 2005-11-21 09:54
Joined: 2005-02-12
Forum posts: 98
Re: File, Streams and Stores
I faced a very interesting problem Although i fixed it but could not know the exact reason.
Problem was in my video ring toner application i used stream store to read and write current setting of application done by user(volume ,video name e.t.c).

My problem is that if
I am using following way to hold setting file location
_LIT(KSettingFilePath,"C:\\setting.txt");
no problem
but if i store setting.txt  in current application folder (C:\\system\\apps\\testplayer\\)
then reading and writing still works fine. but strangely when incoming call arrives call watcher object nowplays video but this time video hangs for one second in middle.

How is it possible?HuhHuh

What is the actual mistake done by me or is it a known issue on symbian(API bug)

I am only chanfging following line

_LIT(KSettingFilePath,"C:\\setting.txt");
to
_LIT(KSettingFilePath,"C:\\system\\apps\\testplayer\\\setting.txt");

and bug is introduced.
Plz suggest me..
Thanks

Tue, 2006-02-28 14:20
Joined: 2005-02-01
Forum posts: 36
Re: File, Streams and Stores
triple backslash:
_LIT(KSettingFilePath,"C:\\system\\apps\\testplayer\\\setting.txt");
Wed, 2006-03-01 05:51
Joined: 2005-02-12
Forum posts: 98
Re: File, Streams and Stores
that is just typo while posting not in actual code.well thanks for reply
Thu, 2006-03-09 11:58
Joined: 2005-06-12
Forum posts: 41
Re: File, Streams and Stores
/* Txt file to be opened. */
   RFile file ;
   file.Open(aFs, aFilename,EFileRead) ;
////////////   CleanupClosePushL(file);

   /* Size of file. */
   TInt iSize ;   
   file.Size(iSize) ;
   
   /* Allocate enough memory to read in the whole file. */
   HBufC8* buf = HBufC8::NewL(iSize) ;

   /* Get a pointer to modify contents of buffer. */
   TPtr8  pBuf = buf->Des() ;

   /* Read data into buf. */
   file.Read(pBuf) ;

   ////////////CleanupStack::PopAndDestroy( //; // file

   file.close();





ls the above  program correct? I have just replaced  CleanupStackPushL & PopandDestroy statement by a file.close statement at the end ?


Thu, 2006-03-09 22:12
Joined: 2006-03-04
Forum posts: 194
Re: File, Streams and Stores
>> I have just replaced  CleanupStackPushL & PopandDestroy statement by a file.close statement at the end ?

But why? What are you hoping to achieve?
If you don't call CleanupClosePushL() and HBufC8::NewLC() leaves then what happens to your file handle?

Mon, 2006-04-17 09:15
Joined: 2006-04-04
Forum posts: 4
Stores - CRichText::ExternalizeL(...) does not work
I would like only to add here one thing concerning stores with which i had problems:

CRichText::ExternalizeL(...)  does not work !!!

i found lots of posts from people that had problems with this ... and nobody made it work correctly (all the time it throws some undocumented panic which i was not able to find in symbian include headers)

create a new store instead and to save datam fro rich text use

CRichText::StoreL(...)
  • Login to reply to this topic.