Some time ago, I asked several people about khmer writing styles and here is the most documented answer, by Michel Antelme :
If you have a look at school textbooks from the 1950s or 1960s, "Khmer Os Muol" corresponds to aksar kham (អក្សរខម) and "Khmer OS Muol Pali" to aksar mul (អក្សរមូល).
Please do have a look at Guesdon's Dictionnaire cambodgien-français, Paris, Librairie Plon, p. 10* and 11* (first two pages of the attached document). What he calls "Mul" is obviously identical to "Khmer OS Muol Pali". You'll also see in Guesdon's dictionary that the foot (choeng) for da (ដ), in the two styles he gives, i.e. the Mul and the Chrieng scripts, was different from the choeng of ta (ត). It was not frequently used, and when used, only for Sanskrit and Khmer words. It was still used in some later publications, but is unfortunately absent from the Buddhist Institute's Dictionnaire cambodgien.
The third page of the attached document is the scan of a photocopy from an old textbook. You'll see that the script is called Aksar kham (អក្សរខម). It corresponds to the style commonly used nowadays in street signs. Kham is the name by which the (Siamese) Thais and the Laos called the ancient Khmers (and is most probably from the word krom meaning below or downstream which was used by the Mons to call the Khmers).
The 4th page (pp. 52-53) is from Derek Tonkin's How to write the Khmer language. The Cambodian Alphabet, Phnom Penh, Université Buddhique Preah Sihanouk Raj, 1961. What he calls "Text in Round Script" in English and អក្សរមូល in Khmer is exactly the same font used by Guesdon. You can see that the "Khmer OS Muol Pali" is directly inspired by this font.
In handwriting, both syles are not distinguished by their boldness. They're nearly identical. See for instance Franklin Huffman's Cambodian System of Writing and Beginning Reader (5th to 7th pages of the attached document). The only obvious differences are for letters ក (ka) and its foot, ជ (cho) and its foot, and វ (vo). The feet of ន (no) and រ (ro) can also diverge between the two scripts. Huffman doesn't give the feet in his table but you can find them in Guesdon's dictionary.
So clearly, if we want to be accurate, "Khmer Os Muol" is aksar kham (អក្សរខម) and "Khmer OS Muol Pali" is aksar mul (អក្សរមូល). In colloquial speech, both scripts were called "mul" so it could be -- and actually was -- confusing. But if you want to use proper and correct names, "Khmer OS Muol Pali" shouldn't be called aksar kham (អក្សរខម).
That was before the Khmer Rouge period. Nowadays, most people, especially the younger generations don't know anything about these names, so it got all mixed up. Let's try as scholars to restore proper names. Apart from this computer-font graphic designers can call the styles they create as they like.
P.S. Aksar kham (អក្សរខម), is said to be a variant used in Siam to write Pali and Siamese Thai (see pages 8 to 11 of the attached document), only for religious works and it differed very slightly from the round script used in Cambodia. Both variants, the Siamese one (called Khmer script) and the Khmer one called the round script are both directly inherited from the Angkorian script. Later on, the Khmers also adopted the Siamese Kham script which, in the 20th century, became more common than the Mul script. In the last page of the attached document is an excerpt from a text in Thai language but written in Khmer characters. As you can see two tonal signs are used, the ' for the first tonal sign, and the second tonal (which I write on the Thai letter kor) ก้. The third and fourth tonal signs are missing from this excerpt as they're not that common, being used mainly with onomatopoeaic words. Moreover, the Khmer script used in Thailand has two signs for consonant /f/ (ฝ and ฟ in Thai script) and also a consonant sign (fot Thai ซ). I hope that one day all these signs will be encoded for Unicode Khmer characters as the Khmer script was not only used for Khmer, but also Sanskrit, Pali AND Thai.
Please, find also attached a page from Guesdon's dictionary, a table of independant vowels. It's interesting as you can see that for au ឳ, there are two different shapes. I can't write the second one as it's lacking from Khmer OS, but I've already seen it in palm-leaf manuscripts. It's to be distinguished from ឲ. I mention it as in some very recent fonts ឲ (in ឲ្យ) is sometimes written with the sign resembling the variant of ឳ, which is a mistake.
The other page is about figures. As you can see 9 had two differents forms. The one which is no longer used was considered by some to be the Khmer figure, whereas the one which is nowadays commonly used was said to be "Siamese".
I also have found these sources, confirming this:
If you have other valuable sources, I'd be very glad to add them here.