| 10-29-2006, 03:05 AM | #1 |
I remember pitzermike saying something on the subject but my memories are fuzzy right now. If my program only parses and writes JASS code - it doesn't show it in a windows control or in anyway in the screen - Is it necessary to decode UTF8 from the input and encode it for the output? |
| 10-29-2006, 03:57 AM | #2 |
I think so, because if you look at it one byte at a time you might find a quote or '\n' or other syntactically relevant thingy inside one of the wide characters. |
| 10-29-2006, 10:13 AM | #3 |
We might have to decode it in strings yes.. We skip comments so we don't have to decode anything there. |
| 10-29-2006, 08:48 PM | #4 |
The way you skip comments is by going from // to '\n'. If your parsing is one byte at a time, gg. |
| 10-30-2006, 08:26 PM | #5 |
According to the UTF8 spec 'code points' that use more bytes than 1 contain bytes bigger than 127, so we only need to decode for showing up to the screen and it actually seems that pitzermike was right. There is no way '/' or '\n' can appear in an UTF8 string and not mean what they mean on ascii. |