ucol_getSortKey may ignore buffer's length

• Thu Oct 19 23:54:47 2006 grhoten sent reply 1
• Mon Oct 30 02:00:50 2006 weiv sent reply 2
• Mon Oct 30 03:44:48 2006 weiv sent reply 3
• Mon Oct 30 03:45:19 2006 weiv changed notes2: assign: "" to "weiv", priority: "" to "high", target: "UNSCH" to "3.8", weeks: "" to "0.5",
• Mon Oct 30 03:45:19 2006 weiv moved from incoming to collation
• Mon Oct 30 19:01:58 2006 guest sent reply 4

Here is an example from glib patches:

-+ uint8_t dummy;

+ int32_t result_len; + + /* get size of result */

-+ result_len = ucol_getSortKey(icu_collator, wstr, wstr_len, &dummy, 1); ++ result_len = ucol_getSortKey(icu_collator, wstr, wstr_len, NULL, 0);

Hi Mi,

it would be very useful if you could provide me with a code snippet that fails. It is kind of hard to find this problem without an example that tackles it.

Thank you very much!

Regards, v.

Mikhail,

Yes - I found the problem. It can bite you if you pass in buffers of size 1 or 2 and it will almost always bite you, since I can write 1 or two bytes of primary key before I check whether there is enough space in the buffer. Sorry about that.

I will look into fixing this. For right now, I recommend either: - passing in a zero buffer, like you did in your fix. This solution is as good performance wise as the one with the dummy buffer, and better - since it doesn't crash. The problem with this solution is that you have to calculate the contents of the sort key twice - first time to get its length and then the second time to fill the buffer. - better performance wise would be establishing a buffer of reasonable size (depending on the contents of your string between 3 and 4 times the number of characters) and passing that. This way there is a reasonable chance that you will get the sort key in the first call of ucol_getSortKey.

*** For the bug ***

ucol.cpp in ucol_calcSortKey has an initial bailout to ucol_getSortKeySize in the beginning for resultLength == 0 primaries == NULL (ucol.cpp:4530 release-3-6). If a dummy buffer of size 1 or 2 is passed in, we will continue to calculate the key and in case of a non-ignorable primary we will write 1 or 2 bytes to this buffer *before* we check if the buffer is full. There are several ways to fix it. For right now:

*** WORKAROUND *** Do not use buffers of size 1 or 2 for calling ucol_getSortKey.

Not that you should ever do this...

Hope this helps.

Regards, v.

On Oct 29, 2006, at 7:15 PM, Mikhail Teterin wrote:

A snippet from glib's code is now attached to the Problem Report, is not it? In the past (before patching as per that attachment), building glib2 with ICU and trying to start gimp, for example, would cause the application to crash :-(

-mi

(Guest Reply)

Btw, I initially used a 1 byte buffer because the API doc did not specify that null was allowed. Maybe you'll want to update the doc.

I seemt to have a similar Problem woth ucol_getSortKey.

I'm on icu4c 3.8, on a ubuntu dapper box

I use icu4jni to get a sort key for a source string. Following Code in Java will lead to a stack corruption that even makes text in the current terminal look weird:

Collator mColl = Collator.getInstance(); mColl.setStrength(Collator.PRIMARY); mColl.getCollationKey("2d294f2d3739433565147655394f3762f3147312d3731641452f310");

The corruption happens during the call to ucol_getSortKey in CollationInterface.c of icu4jni.

could it be possible that this is caused by this/a similar bug?

I might have to check this again ... i'm not so sure anymore why this corruption happens. i hope it was due to a mistake i made and not because there's a bug in the icu method. If this can be deleted, please do so. I'll post again if i am sure where the problem lies. Or not if the mistake happened on my side.

Replying to ursgrob(at)rubbernose.ch:

I seemt to have a similar Problem woth ucol_getSortKey. I'm on icu4c 3.8, on a ubuntu dapper box I use icu4jni to get a sort key for a source string. Following Code in Java will lead to a stack corruption that even makes text in the current terminal look weird:

I recommend that you use ICU4J instead of ICU4JNI. ICU4JNI is no longer being maintained, and ICU4J has all the functionality you need. If you have further questions, you should use the icu-support mailing list.

Please review the ChangeSet24155:http://bugs.icu-project.org/trac/changeset/24155

Note, the ABW problem deos not only happen in small key buffer.