Fixing Korean input in 9front ktrans

, updated

한국어: 9front ktrans 한글 입력 고치기

ktrans, the input method in 9front, has a Korean mode. It was there, though it had been left broken and could not really be used. I generated a new hangul.map and wrote lookup code for Dubeolsik input, and the patch was merged into 9front in April 2024.

Before the fix

The ktrans(1) manual had the following description for Korean input:

Mapping is done by emulating a Dubeolsik layout, with each Latin
character mapping to a single Jamo. Sequences of up to three Jamo
are automatically converted to Hangul syllables.

It says it emulates a Dubeolsik layout and automatically converts up to three Jamo into a syllable. In practice, gksrmf (한글) worked, but rk (가) stayed rk, and tkfkd (사랑) became 살ㅏd. zhem (코드) did not work either. A single consonant (r) did not come through, while a single vowel (k, ㅏ) did.

First of all, /lib/ktrans/hangul.map was not Dubeolsik. hangul.map is a table with key TAB character on each line. Out of the 11,172 Hangul syllables in Unicode, it contained only the 2,350 syllables from KS X 1001, and even those had unexpected Shift keys assigned to compound vowels and compound final consonants. In Dubeolsik, '과' is rhk, but the old file had this:

rK	과
rH	괘
rL	괴

The line order of the file was also a problem. ktrans operates by reading the map file from top to bottom and overwriting every prefix of each key with an entry that produces no output yet and waits for the next key. The old file registered shorter keys first. Even after rk 가 was registered, a later line like rkr 각 overwrote rk with a non-output entry. Most syllables without a final consonant and all single consonants were erased in this way. Because no syllable starts with a vowel, only single vowels survived. Japanese and Chinese modes have ctl-\ to convert the current input manually, but Korean mode was not connected to it.

Even if the map file had been intact, the lookup code could not handle Hangul. With no code dedicated to Korean, it was using the same generic code as Japanese. The generic code collects input keys and commits a character only when no further key can follow. There is no mechanism to carry a final consonant over as the initial consonant of the next syllable. Typing tkfkd (사랑) ends at tkf as '살' because no further key can follow it, leaving the k and d behind in pieces.

New map

Unicode's 11,172 Hangul syllables are arranged in order starting from 0xAC00 ('가'), across 19 initial consonants, 21 medial vowels, and 28 final consonants (including no final).

I wrote genmap.go to obtain the initial, medial, and final indices for each syllable and join the corresponding Dubeolsik Roman keys. Compound vowels and compound finals also use two keys just as in Dubeolsik. ㅘ is hk, and ㄺ is fr.

	for r := 0xAC00; r <= 0xD7A3; r++ {
		cho := choseong[(r-0xAC00)/(28*21)]
		jung := jungseong[(r-0xAC00)/28%21]
		jong := jongseong[(r-0xAC00)%28]
		if jong == "" {
			fmt.Printf("%s%s	%c\n", cho, jung, r)
		} else {
			fmt.Printf("%s%s%s	%c\n", cho, jung, jong, r)
		}
	}

Adding 19 lines for initials and 21 lines for medials for typing standalone Jamo after the 11,172 syllable lines generated 11,212 lines in total.

So that longer keys do not overwrite the output of shorter keys when ktrans reads the map, longer keys have to come first. I sorted the file in reverse order of Roman key length to produce hangul.map.

go run genmap.go | awk '{printf "%d%s\n", length($1), $0}' | sort -nr | sed 's/^[0-9]//' > hangul.map

With this order, rkr 각 is read before rk 가, so rk keeps '가' as its output while waiting for the next key.

Lookup code

In the new map, key prefixes like z (ㅋ), zh (코), and zhe (콛) all correspond to characters as well. The existing generic code sends the Roman character to the screen first whenever a key is typed, and once a match is found, sends as many backspaces as the number of collected keys to erase them before outputting the converted character. If the prefix has already been combined into a single character on screen, too many backspaces are sent, erasing the preceding character as well.

For Hangul input, I wrote dubeollkup to handle key input and dubeolbksp to handle backspace separately. Only the keys of the single syllable currently being composed are kept in the line buffer.

When a key is typed, the end of the screen has the syllable being composed followed by the single English key just typed. If there is only one key in line, only the key just typed needs to be erased (1 backspace); if there was a syllable being composed, both that syllable and the key just typed need to be erased (2 backspaces).

With the new key appended to line, dubeollkup first checks whether the whole line is in the map. When it is, as in going from z (ㅋ) to zh (코) and zhe (콛), it sends the backspaces and outputs the new syllable.

If the whole line is not in the map, but the last two keys (buf) are, the final consonant of the previous syllable carries over as the initial consonant of the new syllable. When m (ㅡ) is typed in the state zhe (콛), zhem does not exist in the map, but em (드) does. In this case, the last two keys are popped from line to output the previous syllable again without its final (zh, 코), followed by the new syllable (em, 드). Only em remains in line. Compound finals split in the same manner. Typing k (ㅏ) after rkfr (갉) carries rk over to become '갈가'.

If neither applies, composition of the previous syllable has ended, and line starts over with the single new key.

	if(hmapget(hangul, line->b, &lkup) < 0){
		if(hmapget(hangul, buf, &lkup) < 0){
			resetstr(line, nil);
			line->p = pushutf(line->p, strend(line), e1, utflen(e1));
			if(hmapget(hangul, line->b, &lkup) < 0)
				return;
		}else{
			/* treat Jongseong as Choseong when it matches with new Jamo */
			popstr(line);
			popstr(line);
			hmapget(hangul, line->b, &lkup);
			emitutf(output, backspace, 2);
			emitutf(output, lkup.kana, 0);

			hmapget(hangul, buf, &lkup);
			emitutf(output, lkup.kana, 0);
			line->p = pushutf(line->b, strend(line), buf, utflen(buf));
			return;
		}
	}
	if(utflen(line->b) == 1)
		emitutf(output, backspace, 1);
	else
		emitutf(output, backspace, 2);
	emitutf(output, lkup.kana, 0);

Backspace is also passed to the screen first like other keys, erasing the last character. dubeolbksp, called afterward, pops the last key from line and outputs the character corresponding to the remaining keys back to the screen.

static void
dubeolbksp(Str *line)
{
	Map lkup;

	popstr(line);
	/* lookup and emit remaining Jamo to output */
	if(hmapget(hangul, line->b, &lkup) < 0)
		return;
	emitutf(output, lkup.kana, 0);
}

Pressing backspace after typing '코드' erases '드' on the screen, and line goes from em to e, outputting 'ㄷ' to the screen again. Typing k next makes it '코다'. Preceding characters whose composition has already ended are no longer in line, so they are erased one syllable per backspace.

I added a test to test.c to verify final-consonant carryover and backspace. \b is backspace.

gpffhdnjfemsms wlruqwksw\bgdk, wha tlstjsgks rj djqtdj?
헬로월드는 지겹잖아, 좀 신선한 거 없어?

Typing up to wksw produces '잕', backspace reverts it to '잔', and typing g makes '잖'.

Patch and usage

I sent the patch to the 9front mailing list on March 24, 2024 (mailing list thread), and it was committed on April 2 (68512a3f6). It has been included since the 9front-10522 release later that month.

Run it in a rio window as follows:

ktrans /dev/kbdtap

Pressing ctl-s switches to Korean input, and ctl-t switches to English input. To start in Korean from the beginning, pass the -l ko flag. Except for ㄲ, ㄸ, ㅃ, ㅆ, ㅉ, ㅒ, and ㅖ, all Jamo are typed without Shift, just as in Dubeolsik.