Some characters appear as boxes with numbers in them

[PSyMastR]

Under the normal Windows binary of Pidgin, Asian characters are not displayed on the buddy list. When hovering over a screen name however, the contact window that pops up shows the characters just fine. This happens on Windows XP and Vista.

[rlaager]

Which protocol?

[wake]

Both my QQ and MSN do not display chinese/Korean/Japanese characters correctly. They only show up as a block with some numbers inside. However, they do show in tooltips.

[PSyMastR]

Hmm the problem seems to be fixed with the 2.0final release of Pidgin on XP. I have yet to test Vista yet with this. But yes, It was the same problem that wake was describing.

[cuivien]

downloaded version 2.0.0 today from sourceforge, problem is not fixed on my computer.. WinXP, newest version of gtk+ (2.10.11) and everything else.

Characters are only visible on hovering in contact list and in actual conversation. (however, in conversation window, only stuff written by someone else is visible, not my own text or kanji and stuff in nick)

[datallah]

Replying to cuivien:

downloaded version 2.0.0 today from sourceforge, problem is not fixed on my computer.. WinXP, newest version of gtk+ (2.10.11) and everything else. Characters are only visible on hovering in contact list and in actual conversation. (however, in conversation window, only stuff written by someone else is visible, not my own text or kanji and stuff in nick)

Did you install the new GTK runtime from 2.0.0 final?

If not, replace your Program Files\Common Files\GTK\2.0\etc\pango\pango.aliases with this one.

[cuivien]

replacing alias file does not work. reinstalling and installing gtk from 2.0.0. does not work either

[59ideas]

I am also having this problem after install from 2.0.0 my pango.aliases looks exactly like the one from http://pidgin.im/~datallah/pango.aliases

[OCedHrt]

This is definitely not working. I can confirm on Vista as well.

Why is it broken again? This issue has been raised several times before and was fixed way before 2.0 final.

[datallah]

Replying to OCedHrt:

Why is it broken again? This issue has been raised several times before and was fixed way before 2.0 final.

This has never been "fixed" because the root problem is that Pango doesn't deal with font fallbacks correctly.

[datallah]

Related to #191

[datallah]

The fix is going to involve adding more fonts (that contain the appropriate unicode glyphs) to pango.aliases. If someone knows the names of the default Windows CJK fonts in the various font families, I'd be happy to add them to the lists.

[OCedHrt]

I do not have the specific list of CJK fonts with me right now, but Arial Unicode MS could be listed as one of the fail safes (would need to have office installed).

However, the CJK fonts would only work for CJK, if anything else is missing, they would still be broken.

[carlthuringer]

Japanese text replaced by squares

[carlthuringer]

This is extremely important to me as a student of Japanese. I can't communicate with people using pidgin, which means I have to install AIM or MSN and run that alongside, which is just inefficient and confusing.

When/how can this be fixed? I was looking forward to 2.0.2 and it didn't make that milestone.

[cheungpat]

For the time being, try to disable custom font in the messaging window. (Option available in the Preference dialog box) I am able to see the Chinese characters in the new incoming messages. Replying to carlthuringer:

When/how can this be fixed? I was looking forward to 2.0.2 and it didn't make that milestone.

[OCedHrt]

CJK characters work fine for me in the IM message windows. It's only on the buddy list where I have the problem.

[carlthuringer]

Disabling font faces for incoming and outgoing messages worked. I lose one feature to gain another, but until fallback fonts for special characters come in then I suppose I will have to stick with the default.

[ynotswim]

I can't display Chinese characters in the buddy list, nor chat windows. Anybody knows how to make it work? I am using Pidgin 2.0.2 running on Win XP

Debugging Information

Plugins: Enabled SSL: SSL support is present. GTK+ Runtime: 2.10.11 Glib Runtime: 2.12.11

Library Support

Cyrus SASL: Disabled Gadu-Gadu library (libgadu): Internal GtkSpell?: Enabled GnuTLS: Disabled Network Security Services (NSS): Enabled Perl: Enabled Tcl: Enabled Tk: Enabled

[Pho House]

Hi everyone,

I had problems displaying characters in the chat window, but not the buddy list. When I typed Japanese, I only got weird boxes, but the recipient could see my Japanese words correctly.

I fixed this by going to "Preferences>Conversations Tab" and checked "Use Font From Theme". U have to restart the current chat window, but not the Pidgin program.

Version: Pidgin 2.0.2

I hope this helps!

[Pho House]

k, I just figured that if u don't want to use the font from the themes, you'd have to use fonts within the list that is compatible with Japanese. Go to "Preferences>Conversation" tab, click on "Conversation Font", pick and font and type some Japanese in the test area. If it shows up, you're good to go.

Hope this helps!

[hiddensanctum]

So when are they going to fix this problem? it still exists in Version 2.10. Although the problem for the im window can be solved, there doesn't seem to be a way to make it work on the buddy list

[trev.stephenson]

ok I'm not exactly sure what the problem is but you can work around it by changing the default GTK theme.

This can be done by running "C:\Program Files\Common Files\GTK\2.0\bin\gtkthemeselector.exe" and setting the default theme to something else like "Clearlooks".

After you've done that restart pidgin and you should see your chinese/japanese characters appearing in the main contact list window.

[ynotswim]

Nope. I have the theme set to Clearlooks, and it still doesn't show the characters.

I guess I am not so sure how the font works.

On my Windows XP, I have "Text Services and Input Languages" set to only EN under "Regional and Language Options," because I can't stand the interruption by Microsoft IME. But I am still able to read/view other characters in a Word document or on web sites.

Does pidgin rely on this setting? If so, in what way?

[hiddensanctum]

Your solution works for me (running on Vista) Thank you!!

[adamsonh]

I tried all the methods listed above, but I still could not solve the problem. I have no problem with most of my contacts who use Chinese (both simplified and traditional). Only 2 out of them have such a problem, but log files look OK. This morning I installed pidgin 2.11 onto a simplified Chinese Windows 2000 computer and found the same problem with those 2 users. Please pay some attention to my posts. I can ask those two users to provide information useful for debugging.

[datallah]

Replying to adamsonh:

I tried all the methods listed above, but I still could not solve the problem. I have no problem with most of my contacts who use Chinese (both simplified and traditional).

The fact that you're seeing the text correctly for most people means that the problem isn't the issue that this bug report was originally created for.

Only 2 out of them have such a problem, but log files look OK. This morning I installed pidgin 2.11 onto a simplified Chinese Windows 2000 computer and found the same problem with those 2 users.

Please elaborate exactly what "the problem is" - is it that the buddy's name doesn't appear in the buddy list? Does the conversation text not appear? Which protocols (ICQ, MSN, AIM) are these buddies using?

[adamsonh]

Replying to datallah:

Replying to adamsonh:

I tried all the methods listed above, but I still could not solve the problem. I have no problem with most of my contacts who use Chinese (both simplified and traditional).

The fact that you're seeing the text correctly for most people means that the problem isn't the issue that this bug report was originally created for.

Only 2 out of them have such a problem, but log files look OK. This morning I installed pidgin 2.11 onto a simplified Chinese Windows 2000 computer and found the same problem with those 2 users.

Please elaborate exactly what "the problem is" - is it that the buddy's name doesn't appear in the buddy list? Does the conversation text not appear? Which protocols (ICQ, MSN, AIM) are these buddies using?

Please go to #2664 for my original post. Someone thought #2664 was the same as #604 and closed it.

[daeboe]

I looked up above and saw resolution fixed, yet I don't see any solution to this problem. I have this problem too, however I have traditional chinese contacts instead of simplified. I tried the listed above methods, none of them work. Although if I run msn, or yahoo, I can read all those characters fine. both my contacts are in yahoo/msn.

[daeboe]

Chinese contact missing proper characters, subbed out by squares with more squares in them.

[QuLogic]

Replying to datallah:

The fix is going to involve adding more fonts (that contain the appropriate unicode glyphs) to pango.aliases. If someone knows the names of the default Windows CJK fonts in the various font families, I'd be happy to add them to the lists.

Are you still looking for this?

According to Fontboard, the available CJK fonts installed with various Office/Internet Explorer language packs should be, in no particular order:

Traditional Chinese: MingLiU, PMingLiU
Simplified Chinese: SimSun, NSimSun, SimHei, MS Song, MS Hei
Japanese: MS Mincho, MS PMincho, MS Gothic, MS PGothic, MS UI Gothic
Korean: Batang, BatangChe, Gungsuh, GungsuhChe, Gulim, GulimChe, Dotum, DotumChe

I have no idea which ones look "better" or "worse", or what their coverage is. Also, 'Arial Unicode MS' is supposed to cover most of the Unicode specs.

Microsoft also has a list of fonts, but you can't search in reverse (by Unicode block->Font->Provider), so I didn't bother to look very much.

[daeboe]

Replying to QuLogic:

I have no idea which ones look "better" or "worse", or what their coverage is. Also, 'Arial Unicode MS' is supposed to cover most of the Unicode specs. Microsoft also has a list of fonts, but you can't search in reverse (by Unicode block->Font->Provider), so I didn't bother to look very much.

I can read Chinese at the very least, and tell the differences between the Japanese and Korean characters. If you need any help or whatnot, just let me know. I'd be glad to explain to you what looks "correct" in those terms. But personally, I'd say just download all the fonts or character sets, and get support for those fonts. Pretty much just release a font package for Pidgin.

[QuLogic]

Replying to daeboe:

I can read Chinese at the very least, and tell the differences between the Japanese and Korean characters. If you need any help or whatnot, just let me know. I'd be glad to explain to you what looks "correct" in those terms. But personally, I'd say just download all the fonts or character sets, and get support for those fonts. Pretty much just release a font package for Pidgin.

No, I doubt there would be a font package for Pidgin. The relevant fonts would need to be installed from official language packs (and presumably have already been installed, anyway). Datallah would just do as he said:

The fix is going to involve adding more fonts (that contain the appropriate unicode glyphs) to pango.aliases.

[ajmas]

From what I can see it is possible to change the font for the conversations, via:

Preferences -> Conversations -> Conversation Font

I have changed mine to a font that supports the complete unicode range.

The problem is that the default font is not a unicode complete font and there is also no way to change the font for the 'Buddy List', so I am just getting the missing characters boxes. Using one of the standard Windows XP fonts, that is Unicode complete IMHO would be the best approach. Mixing and matching fonts that support different ranges is more complicated.

This observation is from using Pidgin 2.3.1 on Windows XP.

[ajmas]

Replying to trev.stephenson:

ok I'm not exactly sure what the problem is but you can work around it by changing the default GTK theme. This can be done by running "C:\Program Files\Common Files\GTK\2.0\bin\gtkthemeselector.exe" and setting the default theme to something else like "Clearlooks".

The problem is that you shouldn't need to make any manual changes to get international support. The reason it is a problem is because most people expect things to 'just work' and manual tinkering of configuration files is usually only considered to be for power users.

[TravisM]

I too am having this problem, it is fine when I receive messages from Gmail, but I can't read them from a MSN user. Running XP, and the latest Pidgin and GTK files. Any ideas?

[blackwizard]

Is there a fix for this problem? I am having similar issue.

I'm currently on Korean Windows XP, with Pidgin set to Korean locale. Buddy list is showing up fine as near as I can tell (although I am not ENTIRELY sure on this, since I reset all my contacts' names, so I might have missed something along the way), and I can RECEIVE messages in Asian languages without having them getting broken like those screenshots above.

HOWEVER, whenever I try to type in Japanese, characters whose glyph doesn't exist within Korean font are broken up. The strange thing is that when I send the message, my message still appears broken, but people receiving my messages can see it just fine.

I know that this can be considered a minor thing, but it really is driving me up the wall every time that I need to use Japanese.

I've tried ALL of the suggestions above, but no luck. Is there a definite fix for this?

[datallah]

Ticket #6974 has been marked as a duplicate of this ticket.

[puzzles]

I tried adding a couple different fonts to my pango.aliases, but this problem just won't go away on Win2k with Hindi/Devanagari. I dug up an old Tor Lillqvist thread archiveon CJK support and Windows with basically the same advice, change pango.aliases. The thread is from 2002, though; unsure if it's relevant.

Pidgin 2.5.2, GTK2+ 2.12.12

Perhaps someone with a better GTK savvy can push this up to the GTK people and word towards a fix.

[hueykam]

I had the same problem with the current Pidgin 2.5.2 with Traditional Chinese Font. The solution to let the font fall back to use MingLiu?.

Here's the detail instructions for slow people (like myself):

1) Edit the pango.aliases (with WordPad?) It's at c:\Program Files\Common Files\GTK\2.0\etc\pango\pango.aliases

2) Add the following lines: arial = "arial,mingliu" verdana = "verdana,mingliu"

3) Restart Pidgin.

Summary: The idea in step 2 is to tell Pango that if the font comes in as arial, use arial, if u can't then fall back to use mingliu etc etc

Bcos most other people just use default Arial to type their messages. Apparently, people can enter Traditional Chinese messages with Arial fonts too, but Pidgin on this side needs to be able to display it.

Similar idea should be applicable for other languages. However, if you still cannot view the message after doing this, try to find out what font the other person sends it in, then do similar addition to step 2.

ie if the font is XXX, then you know YYY font can display it. then use XXX = XXX,YYY

[hueykam]

Btw, I think in earlier version of Gaim, there is a feature to ignore fonts from the sender and use our own font. If we can have this feature, then nobody is going to complain that their language/font is not displaying...

Can I suggest this to be a feature for future Pidgin?

[hueykam]

Replying to hueykam:

2) Add the following lines: arial = "arial,mingliu" verdana = "verdana,mingliu"

This should be in different lines I think
arial = "arial,mingliu"
verdana = "verdana,mingliu"

[ajmas]

Another approach is to use something like Firefox uses. Basically, from my understanding, it will use the font requested, but if the characater is not defined in the given font, then it will attempt to take it from another font. The only question, is whether this logic should be provided by Pidgin, or the display API on which it depends?

[rlaager]

ajmas: That's exactly the suggested fix. And no, it shouldn't be provided by Pidgin. It should be (and is) provided by Pango. In this case, the issue is probably that Pango's default aliases file is incomplete. Additions should be suggested to them.

datallah: Did you see QuLogic's comment (comment 30)?

[ajmas]

Should Pango even need a configuration file? Surely this sort of information could be dynamically calculated, maybe at install time and then whenever the font list changes?

[hueykam]

Damn, this solution does not work if the font is serif. No matter what you put for serif, it just does not response. Any idea why this is the case?

serif = "mingliu"

Times New Romans works if this is added:
"times new roman" = "times new roman,mingliu"

Replying to hueykam:

This should be in different lines I think
arial = "arial,mingliu"
verdana = "verdana,mingliu"

[hueykam]

Found this feature. It's at Preferences -> Conversation Tab. Uncheck the first checkbox "Show formatting on incoming messages". This will display your own font instead of the font that the other side specify.

Replying to hueykam:

Btw, I think in earlier version of Gaim, there is a feature to ignore fonts from the sender and use our own font.

[rekkanoryo]

Is there any progress on updating the aliases file for the fonts with Asian glyphs?

[ynotswim]

Replying to rekkanoryo:

Is there any progress on updating the aliases file for the fonts with Asian glyphs?

I believe a later version 2.5.2 I have been using has this fixed - when you download the GTK, it has the font alias correctly added.

However, I don't think the pidgin will pick that font automatically.

You must go to Tools->Plugins->Pidgin GTK+Theme Control 2.5.2

Then configure the GTK+Interface Font, and have to get a font that actually can pick up an Asian font both in the chat window and in the buddy list window.

I ran into the problem that if I pick a font that can displays things correctly in the buddy window, then everything non-English appears to be squares in the chat window, vice versa.

[datallah]

Ticket #8785 has been marked as a duplicate of this ticket.

[khc]

Ticket #9188 has been marked as a duplicate of this ticket.

[mikaelz]

this has been an ongoing problem for 2 years and it's still not fixed. I don't understand. I've tried everything that has been listed and nothing works. I have language support enabled, and yet characters still show up as boxes in chat and in buddy list. if I send this to someone, they will get the character since they are using a program that actually has proper font support.

[hueykam]

Replying to mikaelz:

this has been an ongoing problem for 2 years and it's still not fixed. I don't understand. I've tried everything that has been listed and nothing works. I have language support enabled, and yet characters still show up as boxes in chat and in buddy list. if I send this to someone, they will get the character since they are using a program that actually has proper font support.

Have u tried this: At Preferences -> Conversation Tab. Uncheck the first checkbox "Show formatting on incoming messages". This will display your own font instead of the font that the other side specify.

[mikaelz]

that didn't work by itelf, I then had to change fonts to Arial unicode

I think this is rather annoying, devs need to fix this since not everyone will go through all the trouble to figure out how to change it.

[thezest]

The following is my solution to display Chinese characters (conversation window, incoming/outgoing messages, buddy status, buddy list, etc.) on my Windows XP:

1. Go to Pidgin -> Preferences -> Converstations, choose a Conversation font which supports your character. In my case, it is Tahoma.

2. Right-click on the Windows desktop area, choose Properties, then click on the Appearance tab, and Advanced button.

3. Under the Item dropdown list, change all the items that support Font to the one defined in Pidgin. In my case it is Tahoma.

4. Apply the settings and look at your Pidgin window, it should take effect immediately.

Hope the above helps.

[bernmeister]

I suspect #8917, #9635 and #9950 are duplicates of #604.

[eric.yuenchan]

Dear All,

May someone summarize the ultimate solutions? i tried a few stated above but still not working. And will it be officially fixed in the coming version of Pidgin???

Please kindly reply.

Eric Chan

[hueykam]

I have not use Pidgin for a while because I am now using Mac. But have u tried this yet:

Preferences -> Conversation Tab. Uncheck the first checkbox "Show formatting on incoming messages". This will display your own font instead of the font that the other side specify.

Let us know if this work?

[hueykam]

In addition to that, change your font to some Unicode font (eg Arial Unicode) or some other font which supports the language you use.

Replying to hueykam:

I have not use Pidgin for a while because I am now using Mac. But have u tried this yet: Preferences -> Conversation Tab. Uncheck the first checkbox "Show formatting on incoming messages". This will display your own font instead of the font that the other side specify. Let us know if this work?

[eric.yuenchan]

Thanks Hueykam.

I tried to change the conversation fonts under preferences-->conversations. I also tried to change the fonts setting in the plug-in Pidgin GTK+ theme control, but both are not working.

I think your solution can only fix part of the problem, that's i can chat and see others messagen Traditional Chinese. But my buddy list and others part still can't show the correct character.

Any other solutions?

Of course the traditional chinese version of Pidgin can shows chinese character without problem. May we copy part of the settings in that config file? or replace that one with the english version?

Regards, Eric Chan

[ajmas]

Can any of the Pidgin developers confirm this is a GTK issue and not a Pidgin issue? If it is confirmed to be a GTK issue then we can open up a ticket, otherwise a Pidgin side fix would be welcome.

Note: I am not a developer on either project.

[datallah]

Ticket #9793 has been marked as a duplicate of this ticket.

[darkrain42]

Ticket #9914 has been marked as a duplicate of this ticket.

[datallah]

Ticket #9806 has been marked as a duplicate of this ticket.

[tokkis]

I was able to fix mine.

1) make sure you have the GTK plugin checked/enabled with appropriate fonts added as mentioned above

2) (critical) under Preferences - Conversations, make sure you CHECK the box next to Use font from theme

That should do it. to test it, type in the test box in that gray window.

[davinns]

@hueykam: your solution works for me but only for the chat window. In the message history window, the chars still display as blocks.

Finally I have a solution. Here is my story:

Since only messages from some but not all contacts displayed as blocks, I thought they might set their fonts to which my system doesn't know. By opening the historical message folder (clicking a button on the message history window) I found some HTML files, then I opened one file with a text editor (e.g. notepad), in the file I saw the messages were formatted like this:

<font color="#A82F2F"><font size="2">(15:49:18)</font> <b>My Friend</b></font> <span style='font-family: Microsoft Sans Serif;'><span style='color: #000080;'>some text</span></span><br/>

From above line I knew my friend was using the 'Microsoft Sans Serif' font, then I add it to pango.aliases:

"Microsoft Sans Serif" = "simsun"

Finally it works for both chat window and message history window.

My friend uses Simplified Chinese, so I set it to "simsun". For those who use Traditional Chinese or Japanese or other languages, they may need to set it to other font.

Wish this solution helps.

Replying to hueykam:

In addition to that, change your font to some Unicode font (eg Arial Unicode) or some other font which supports the language you use. Replying to hueykam:

I have not use Pidgin for a while because I am now using Mac. But have u tried this yet: Preferences -> Conversation Tab. Uncheck the first checkbox "Show formatting on incoming messages". This will display your own font instead of the font that the other side specify. Let us know if this work?

[ynotswim]

Replying to ynotswim:

Replying to rekkanoryo:

Is there any progress on updating the aliases file for the fonts with Asian glyphs?

I believe a later version 2.5.2 I have been using has this fixed - when you download the GTK, it has the font alias correctly added. However, I don't think the pidgin will pick that font automatically. You must go to Tools->Plugins->Pidgin GTK+Theme Control 2.5.2 Then configure the GTK+Interface Font, and have to get a font that actually can pick up an Asian font both in the chat window and in the buddy list window. I ran into the problem that if I pick a font that can displays things correctly in the buddy window, then everything non-English appears to be squares in the chat window, vice versa.

Just an update - I just installed Pidgin 2.6.2 (although I am still updating this ticket for 2.0), the display issue will return. I have to repeat these steps and picked up a font (for me, I used Sans) for both fonts under

GTK+Interface Font Conversation Entry

Then the foreign characters will display correctly. You might have to experiment and pick up a font that works for you.

[rekkanoryo]

Ticket #10830 has been marked as a duplicate of this ticket.

[Calanyo]

I recently came across this problem but in an unusual way.

I've never had any problems sending or receiving chinese text until a couple of days ago. I was having trouble reading the chinese because the font was so small, so I tried changing the font size. I did a dumb thing: without looking too hard in the preferences, I checked out the FAQ and edited the gtkrc-2.0 file. I managed to increase the font size but straight away I was unable to receive chinese characters: only boxes with 4 circles in them. This was only the text that was being sent to me: everything else was fine, including the buddy names in the chat window.

Well, that was no good, so I deleted the config file and restarted Pidgin. It still did it. I tried reinstalling: same. I even uninstalled Pidgin, GTK, cleaned out everything in the registry and renamed the .pidgin directory in my user account (wanted to keep logs, etc.). Then rebooted before reinstalling everything. Still no joy!

I then started mucking about with the preferences and got it to display the text in a font that I know supports chinese characters. That changed all the english characters, but any chinese characters were still just squares.

Today, I managed to get it working again by downloading the pango.aliases and adding "Times New Roman" = "simsun" to the begining. I suppose I should have tried one at a time, but I didn't want to keep on pestering my friend to send me chinese. :)

(BTW, I didn't even have an etc directory under GTK/2.0)

Sorry I don't have too many details, but I thought it was so strange that I should write something down about it.

OS/version: Win XP SP3, Pidgin 2.7.2