Conversion bitween 8 and 16 bits descriptors

Login to reply to this topic.
Mon, 2003-12-15 22:38
eric
NewLC AdministratorSymbian AccreditedForum Nokia Champion
Joined: 2003-01-14
 Forum posts: 1890
There is two kind of conversion between 8 and 16 bits constructor. The first type is just a cast where the data itself are not physically changed :

Code:
// Get a iBuf8 from a iBuf16 (data are not modified)
TPtrC8 ptr8(reinterpret_cast<const TUint8*>(iBuf16.Ptr()),(iBuf16.Size()*2));
iBuf8=ptr8;

// Get a iBuf16 from a iBuf8 (data are not modified)
TPtrC16 ptr16(reinterpret_cast<const TUint16*>(iBuf8.Ptr()),(iBuf8.Size()/2));
iBuf16=ptr16;

The second one takes each character and convert it to the other format. The 16-bit to 8-bit conversion may not always succeed in this case:

Code:
// Get a iBuf8 from a iBuf16 (data are modified)
CnvUtfConverter::ConvertFromUnicodeToUtf8(iBuf8,iBuf16);

// Get a iBuf16 from a iBuf8 (data are modified)
CnvUtfConverter::ConvertToUnicodeFromUtf8(iBuf16,iBuf8);

This second method requires to include the utf.h header and to link against charconv.lib.

Many other methods exist, but these four ones are the one I am using.
Eric

Eric Bustarret
NewLC Founder & CEO / Professional Symbian OS Consultant

Thu, 2004-05-20 11:56
chenziteng
Joined: 2004-05-20
 Forum posts: 22
Conversion bitween 8 and 16 bits descriptors
Thanks for your code, Eric, but I think there is a mistake:
    iBuf16.Size()*2
should be
    iBuf16.Size()
or
    iBuf16.Length()*2
 
Wed, 2004-06-09 23:12
streetmentioner
Joined: 2004-05-28
 Forum posts: 52
Or...
Code:
HBufC* ConvertTo16BitLC(const TDesC8& aDes8)
{
   HBufC* myBuf = HBufC::NewLC(aDes8.Length());
   myBuf->Des().Copy(aDes8);
   return myBuf;
}
 
Sun, 2004-06-27 12:58
siemensc55
Joined: 2004-05-23
 Forum posts: 78
a problem
TPtr16* keyword ;   // a pointer to TPtr16, defined as class variable,all variables below are method variables.
TDes* k;            // a pointer to TDes
HBufC16* key = k->AllocL();//I get a pointer to HBufC16
TPtr16 p = key->Des();   //here I get a TPtr16
keyword =(TPtr16*)&p   // I need to store the pointer to the TPtr16 object in “keyword”.Is it the write way to store a pointer to do .
 
Wed, 2004-10-20 06:05
Anonymous (not verified)
Forum posts: 2018
Re: Conversion bitween 8 and 16 bits descriptors
Quote from: eric
There is two kind of conversion between 8 and 16 bits constructor. The first type is just a cast where the data itself are not physically changed :

Code:
// Get a iBuf8 from a iBuf16 (data are not modified)
TPtrC8 ptr8(reinterpret_cast<const TUint8*>(iBuf16.Ptr()),(iBuf16.Size()*2));
iBuf8=ptr8;

// Get a iBuf16 from a iBuf8 (data are not modified)
TPtrC16 ptr16(reinterpret_cast<const TUint16*>(iBuf8.Ptr()),(iBuf8.Size()/2));
iBuf16=ptr16;

The second one takes each character and convert it to the other format. The 16-bit to 8-bit conversion may not always succeed in this case:

Code:
// Get a iBuf8 from a iBuf16 (data are modified)
CnvUtfConverter::ConvertFromUnicodeToUtf8(iBuf8,iBuf16);

// Get a iBuf16 from a iBuf8 (data are modified)
CnvUtfConverter::ConvertToUnicodeFromUtf8(iBuf16,iBuf8);

This second method requires to include the utf.h header and to link against charconv.lib.

Many other methods exist, but these four ones are the one I am using.
Eric
 
Wed, 2004-10-20 06:06
Anonymous (not verified)
Forum posts: 2018
Re: Conversion bitween 8 and 16 bits descriptors
I think there is a bug here...[/b]
 
Sat, 2005-04-09 22:10
shereen_shaaban
Joined: 2005-03-11
 Forum posts: 17
Conversion bitween 8 and 16 bits descriptors
how do i convert from TBuf<> to TBuf8<>? I've tried the below code but for some reason I keep on getting linking errors:
Undefined Symbol: TParse::TParse(void)

I've added charconv.lib and added #include <utf.h>

TBuf<60> a;
TBuf8<60> b;

b.Copy(a);

What could be wrong?

Shereen.
 
Fri, 2005-05-13 07:55
dotcdotc
Joined: 2004-11-15
 Forum posts: 25
converting TDesC8* to TDesc!6*
how can I convert TDesC8* to TDesC16*?


dotcdotc
 
Tue, 2005-06-21 11:19
ptk
Joined: 2005-06-21
 Forum posts: 1
Re: Conversion bitween 8 and 16 bits descriptors
Quote from: chenziteng
Thanks for your code, Eric, but I think there is a mistake:
     iBuf16.Size()*2
should be
     iBuf16.Size()
or
     iBuf16.Length()*2

Using *2 might not be sufficient, because utf8 characters can consist of three and more charactetrs. Think of the euro sign (\x{20ac}) which occupies three bytes.
 
Mon, 2005-06-27 16:45
nikolas
Joined: 2005-06-13
 Forum posts: 68
Re: a problem
Quote from: siemensc55
TPtr16* keyword ;   // a pointer to TPtr16, defined as class variable,all variables below are method variables.
TDes* k;            // a pointer to TDes
HBufC16* key = k->AllocL();//I get a pointer to HBufC16
TPtr16 p = key->Des();   //here I get a TPtr16
keyword =(TPtr16*)&p   // I need to store the pointer to the TPtr16 object in “keyword”.Is it the write way to store a pointer to do .

As far as I know TPtr is already a pointer to a modifiable descriptor. Why do you have an extra level of indirection? Also "TDes* k" declares a pointer to a descriptor... but where does it point to? Does this code work?

If we fall down it's so we can learn to pick ourselves up.

 
Mon, 2005-06-27 16:51
nikolas
Joined: 2005-06-13
 Forum posts: 68
Re: Conversion bitween 8 and 16 bits descriptors
I think the main issue of this topic is actually people's (including myself) lack of understanding about the Unicode standard.

I believe a topic on Unicode and the different encodings would be most appropriate.

Here's a link to get started:

http://www.joelonsoftware.com/articles/Unicode.html

Any volunteers?

Thanks,

Nikolas.

If we fall down it's so we can learn to pick ourselves up.

 
Thu, 2005-07-28 18:14
MiniVan (not verified)
Forum posts: 2018
Re: Conversion bitween 8 and 16 bits descriptors
I would advise chekcing out descriptors.blogspot.com. It details lots of things you may need or want to do with descriptors.

NOTE: The post by Eric is incorrect and solution by chenziteng works.
 
Tue, 2005-08-02 10:03
nikolas
Joined: 2005-06-13
 Forum posts: 68
Re: Conversion bitween 8 and 16 bits descriptors
I've done some reading on the subject of character sets...   Please correct me if I'm wrong.

First Symbian's internal string format is UCS-2/UTF-16 (they are the same thing).

Now I don't know if you're familiar with the Unicode/ISO-10646 standard.  So I suggest you look up the definitions I use...  Wikipedia has an excellent description and of course there's unicode.org.

All code points can be assigned a 16 bit number.  However not all code points map to valid characters.  Some code points are reserved.  You have 2 groups of code points known as high surrogates and low surrogates.  These code points should only be used in pairs of 1 high surrogate and 1 low surrogate.  These surrogate pairs are used to encode the least used characters.  The result is that effectively unicode characters that fall in this category need 2 16-bits code points to be encoded.  Thus some characters need to be represented with 32 bits.  These characters are however relatively rare.

Also about UTF-8.  As you probably know UTF-8 is a "clever" variable length encoding.  However it has been limited to 4 bytes for UCS-2/UTF-16 compatibilty.

I'm currently doing a little "research" on the Unicode/ISO-106406 standard and I'll putting together some presentation at some point which of course I'll make freely available to anyone.

To be continued...

Thanks,

Nikolas.


If we fall down it's so we can learn to pick ourselves up.

 
Mon, 2005-08-08 04:39
Hunter_Plum
Joined: 2005-08-04
 Forum posts: 8
Re: Conversion bitween 8 and 16 bits descriptors
TUint8 a[]={193,247,196,234,0};
TBuf8<4> b(a);
TBuf<2> c;
CnvUtfConverter::ConvertToUnicodeFromUtf8(c,b);
c is supposed to be "流年",it's value are 27969,24180
but is not,the value it contain are 65533,65533
why?
 
Mon, 2005-08-08 11:44
nikolas
Joined: 2005-06-13
 Forum posts: 68
Re: Conversion bitween 8 and 16 bits descriptors
In UTF-8 a character can be encoded as several bytes.

The first byte indicates the number of bytes that will be used to encode the character, the rest actually encode the character.

I suspect you need more that then 2 bytes per character if you are to use UTF-8 to encode those characters.

Have a quick look on Wikipedia to see if that's the problem....

If we fall down it's so we can learn to pick ourselves up.

 
Wed, 2005-08-24 23:03
queycn
Joined: 2005-08-04
 Forum posts: 32
Re: Conversion bitween 8 and 16 bits descriptors

I want to write /read the TBuf  to the Filestream.

My converting code is not working.  Please give me any advice!

HBufC16* TUEmailSettings::Convert8To16BitLC(const TDesC8& aDes8)
{
   //HBufC* myBuf = HBufC::NewLC(aDes8.Length());
    //TPtr ptr = myBuf->Des();
   //ptr.Copy(aDes8);
    //return ptr;

   HBufC16* myBuf16 = HBufC16::NewLC(aDes8.Length());
   TPtrC16 ptr16(reinterpret_cast<const TUint16*>(aDes8.Ptr()),(aDes8.Size()/2));
    myBuf16=ptr16;
   return myBuf16;


// Get a iBuf16 from a iBuf8 (data are not modified)
//TPtrC16 ptr16(reinterpret_cast<const TUint16*>(iBuf8.Ptr()),(iBuf8.Size()/2));
//iBuf16=ptr16;

}

HBufC8* TUEmailSettings::Convert16To8BitLC(const TDesC& aDes)
{
   //HBufC8* myBuf8 = HBufC8::NewLC(aDes.Length());
   //TPtr8 ptr8 = myBuf8->Des();
   //ptr8.Copy( aDes );
   //return ptr8;

   HBufC8* myBuf8 = HBufC8::NewLC(aDes.Length());
   TPtrC8 ptr8(reinterpret_cast<const TUint8*>(aDes.Ptr()),(aDes.Size()*2));
   myBuf8=ptr8;
   return myBuf8;
   //TPtrC8 ptr8(reinterpret_cast<const TUint8*>(iBuf16.Ptr()),(iBuf16.Size()*2));
   //iBuf8=ptr8;
}
 
123next ›last »
  • Login to reply to this topic.