Note
This is an example of why it's so important to put a date on your blog posts! This is one of my earliest posts (can anyone hazard a guess at the date it was written?) and in fact, pre-dates the whole notion of blogs. It was written for a FidoNet echomail area and for download via modem.
I'd love to hear your thoughts in the comments. Especially if it comes in handy for anyone using DOS emulators etc.
Starting with the Basics
Conventional memory is the PC's native type of memory, and can be accessed directly by DOS and applications programs. It can be limited physically by the amount of memory installed (usually on the motherboard) to say 256 or 512 Kb, but normally 640 Kb will be installed.
Most 8086 based PCs (usually known as XT architecture) will have 640 Kb of conventional memory and nothing else; therefore DOS, device drivers, configuration resources (FILES and BUFFERS), Terminate and Stay Resident (TSR) programs and applications must all co-exist in this 640Kb area of memory.
Why Is There a 640Kb Limit in DOS?
Simple answer: there isn't. The original Intel 8086 processor chip (as used in the first PCs) can access 1 Mb of RAM directly, and so can DOS. It was IBM's decision to reserve the area from 640 Kb to 1 Mb for the system BIOS (Basic Input / Output System), adaptor ROMs (for example for Hard Disks and Network cards), as well as areas of memory used for graphics adaptors such as MDA, Hercules, CGA, EGA and VGA.
Because there are so many adaptor cards that can be installed in a PC, it is not possible to predict which areas of this 384 Kb reserved area are being used and which are not. Some manufacturers use a BIOS of 128 Kb (IBM and Compaq notably) though most clone machines have a 64 Kb BIOS.
Thus it is not really possible to produce a machine with a full 1 Mb of conventional memory, though it is possible to use the areas of RAM installed for VGA and Mono graphics modes for additional DOS use, thus extending DOS's 'memory arena' from 640 Kb to 736 Kb. An example of a program to do this is Quarterdeck's VIDRAM which is supplied with their QEMM memory manager.
I've Got a Bloody Big Spreadsheet and I Need More Memory!
A typical cry heard in the early 1980's, which was solved by the invention of Expanded Memory (or EMS). EMS memory is located outside the conventional 1 Mb, which is accessed by a 64 Kb window. In XT architecture machines EMS is usually provided on a special memory expansion card and accessed with a device driver.
The device driver allocated 64 Kb of conventional memory, which can be used to move chunks of data between EMS and conventional memory. Most EMS managers can handle about 16 Mb of memory on 8086 and 80286 machines. It is not possible to run programs (device drivers, TSRs or applications) in EMS memory, as only small chunks are 'paged' in at any one time.
EMS on 80286, 386 and 486 Machines
With the advent of Intel's new processor chips which have inbuilt memory management functions, it became much easier to bypass the DOS 640 Kb barrier. The 80286 can have up to 16 Mb of extended memory to play with and the 80386 / 486 can access up to 32 Gigabytes (32,000 Mb).
Hang on? Extended memory, not Expanded? That's right. Intel added a new mode to their processors starting with the 286, called Protected Mode (the 8086 can only operate in what is now called Real Mode). In this mode the processor can access all of the available memory in the same manner, without having to resort to complicated paging schemes like EMS.
There were three alternatives; scrap EMS and start using Extended Memory, scrap DOS and start using OS/2 (which runs the processor in Protected Mode and can access that whole 16 Mb), or write device drivers that simulated EMS in Extended Memory.
A few people moved to OS/2 version 1, but the number of applications available was very small. OS/2 version 2 which uses the abilities of the 80386 (a third mode called Virtual Mode) to manage memory and multi-task without fear of one program corrupting another, is a much more viable operating system.
Most people didn't want to give up their favourite applications, and their favourite applications wanted EMS. The easiest solution was to write those device drivers.
Why The 80386 So Soon?
The reason the 80286 was never used as much except a faster version of the 8086 is that it has quite a major design flaw in it. You can change the processor from running Real Mode to Protected Mode with a machine code instruction, but you can't change it back!
Eventually motherboard designers came up with a hardware solution, they wired the SHUTDOWN pin of the 286 back to the RESET pin. Then a program can tell the processor to turn itself off, and this will in fact cause the 286 to reset itself into the mode it uses when it is first turned on, which happens to be Real Mode.
Unfortunately, it's not quite as easy as that, as the program has to make sure that the processor will remember what it was supposed to be doing when it switches back into Real Mode, and this involves quite a detailed knowledge of 80286 machine code and behaviour.
Enter the 80386 processor. As well as Real Mode and a Protected Mode which worked properly, it also had one important new ability. The ability to lie. With the 386 it is possible to instruct the chip to pretend that a range of memory addresses are really another range of addresses. When a program accesses a byte of memory in the 'logical' range, the processor will translate this into a different physical address and use that instead.
(Some later 286 systems with better chipsets, for example NEAT and C&T, can perform some of the memory-relocation functions of the 386, but only with the supplied driver programs.)
The 386 can even implement a virtual memory system, where it can declare areas of the 32 Gb address-space to exist not as physical memory, but as say hard disk space, so the most often used areas of memory will be held in RAM and the less often used areas will be held on disk. DOS doesn't use this technique, but OS/2 version 2 does and Windows/NT might when it is released.
As mentioned above, a new mode was also added called Virtual 8086 Mode, which allows the processor to emulate several separate Real Mode computers, each of which can be running its own programs. This is an extremely safe method of multi-tasking. Windows and OS/2 both use this method to run several programs at once, although OS/2 is more stable than Windows because Windows for some reason runs all Windows programs in the same Virtual Machine (it runs DOS programs in separate VMs though). Thus if one Windows program crashes, the whole system will go down.
OS/2 runs each DOS, Windows or OS/2 application in its own VM, offering extremely good protection against badly behaved or badly written programs.
What's Upper Memory Then?
With the advanced memory management features of the 386 (and a few 286s), it is possible to move a section of memory from one address to another (as far as programs are concerned). Using this technique, the unused areas of memory between 640 Kb and 1 Mb can be 'back-filled' with memory from above 1 Mb.
This creates Upper Memory Blocks or UMBs.
This memory looks just like normal conventional memory to DOS, and you can use it for storing programs and data just like memory below 640 Kb. Because some older programs might get upset if the 'top' of DOS memory isn't at 640 Kb, the UMBs are usually treated as a separate pool of memory, accessed with the DEVICEHIGH command in CONFIG.SYS for loading Device Drivers and the LOADHIGH (or LH) command for loading programs.
There is an undocumented feature of DOS which allows the UMBs to be joined to the rest of DOS memory, allowing COMMAND.COM to be loaded high, and this gives more memory to certain programs.
To switch DOS into this memory-allocation strategy you need a small program (such as my AUTOHIGH.EXE) which tells EMM386.EXE to enable this feature.
And the High Memory Area?
The High Memory Area (or HMA) is a 64 Kb area of Extended Memory which can be used from Real Mode because of a quirk in the way the Intel chips access memory.
With 16 bit registers the PC would only be able to access 64 Kb. Not a lot. To access the 1 Mb in the original IBM PC, the 8086 would have needed 20 bit registers. To get around this problem Intel decided to use a segmented memory scheme. The 1 Mb is divided up into 64 Kb segments, but these start every 16 bytes, thus they overlap quite a lot.
The last memory address in the first 1 Mb is FFFFFh, which is expressed in segment:offset form as FFFF:000F (to calculate the actual address, shift the segment left by one hex-digit and then add the offset).
XT architecture machines have address lines to the processor called A0 to A19. These are the 20 bits needed to access 1 Mb. Processors such as the 286 and above have further address lines from A20 upwards to allow them to access the rest of their 16 Mb.
If there was a way to allow use of the A20 line in addition to the A0 - A19 lines in Real Mode, there would be nothing stopping a program accessing the rest of the segment that starts at FFFF:0000 all the way up to FFFF:FFFF.
This would give DOS another 64 Kb of memory to use, without having to switch processor modes between Real and Protected to get at it. This is quite an advantage as switching modes is quite a slow process, and the extra 64 Kb could be used for loading DOS itself (IBMBIO.COM & IBMDOS.COM or IO.SYS & MSDOS.SYS depending upon whether your copy of DOS comes from IBM or MicroSoft).
Because not all motherboards control the address lines in the same way, a routine called an A20 handler must be installed, one that matches the motherboard or chipset in use. DOS's HIMEM.SYS contains several A20 handlers and will load the correct one when it is installed.
What About Extended Memory?
Ok, so we know about Upper Memory Blocks and the High Memory Area, can we use Extended Memory instead of Expanded Memory? I mean, you wouldn't be limited to accessing it 64 Kb at a time, and you wouldn't have to give up 64 Kb of conventional memory for a 'Page Frame' like you do with EMS.
Well, yes. You could access Extended Memory yourself, like old versions of RAMDRIVE.SYS used to do, but there isn't a standard way of checking to see which bits of Extended Memory are being used by other programs. You would have to try to communicate with each program that you suspect might be using some Extended Memory and ask it, using its API (or Application Programing Interface).
This is, to say the least, extremely tedious. To standardise use of Extended Memory, and to give it a proper abbreviation, XMS was devised. This is a management specification for Extended Memory, and allows all programs to enquire about and use Extended Memory in a standard manner.
The normal way to manage Extended Memory is with an XMS memory manager (or XMS provider) like DOS's HIMEM.SYS or Quarterdeck's QEMM.SYS
MicroSoft in particular has gone XMS mad, if you want EMS on your 386 these days, you have to load HIMEM.SYS first to convert your Extended Memory into managed XMS (HIMEM also loads an A20 handler if possible to convert 64 Kb of your Extended Memory into the High Memory Area), then load EMM386.EXE to emulate EMS in XMS. EMM386.EXE is also DOS's UMB provider, though you can have UMBs without having EMS and vice versa.
QEMM combines the features of HIMEM.SYS and EMM386.EXE and allows dynamic allocation of memory, thus your free memory is in limbo until a program asks for it using either the XMS or EMS APIs. If you have 2 Mb of managed memory, you can allocate all of it as EMS, all of it as XMS or as a bit of both.
As programs free up memory, you can then use it again for either of the memory standards.
Windows requires XMS, SmartDrive requires XMS, RamDrive requires XMS. Unfortunately, older applications and games tend to still require EMS and don't know about the newer and rather more sophisticated XMS specification.
What Version of DOS and What Sort of Memory Should I Have?
Consult the following table for some guidelines:
| Processor | DOS Version | Memory Type | Where | Software Driver |
|---|---|---|---|---|
| 8086 / 8088 | 3.30 or 3.31 | Expanded | Add on card | H/ware specific |
| 80286 | 3.30 or 3.31 | Expanded | Motherboard | H/ware specific |
| 80286 | 5.00 + | Extended | Motherboard | DOS / QEMM etc |
| 80386 | 5.00 + | Extended | Motherboard | DOS / QEMM etc |
| 80486 | 5.00 + | Extended | Motherboard | DOS / QEMM etc |
The Memory Type column shows the type of Hardware memory installed, 386s can emulate Expanded memory in Extended (XMS) memory.
DOS version 3.31 is an OEM version supplied by Compaq and Tandon (and possibly a few other manufacturers) that can support hard disks above 32 Mb. If you can't get hold of it, use 3.30 with Ontrack's Disk Manager if the hard disk is bigger than 32 Mb.
If you can't get hold of DOS 3.3 for an 8086 or 80286, then use DOS 5.00 + rather than DOS 4 which is notoriously flaky.
DOS 5.00 + refers to MSDOS or PCDOS version 5.00 or higher. It does not mean DRDOS 5 (which is a DOS 3 compatible product) or DRDOS 6 which is not command-compatible with any official version of DOS.
While You're At It, How About VCPI and DPMI?
The Virtual Control Program Interface and DOS Protected Mode Interface are commonly known as DOS Extenders. DPMI effectively replaces VCPI. These allow programs to run in Protected Mode without having to change to another operating system, allowing both Code and Data to reside above 1 Mb. XMS and EMS only allow data to be stored above 1 Mb and the PC remains in Real Mode at all other times.
Programs written to use DOS extenders must load a VCPI or DPMI server program before they can run. Some memory managers such as QEMM support VCPI (but not currently DPMI I think), DOS's memory managers do not, so you may have to install a device driver, or even buy a runtime copy of the DOS extender.
Advanced DPMI servers like Borland's are royalty-free (so programmers can include them with their applications) and automatically load themselves when the application program starts up.
What Is Memory Parity?
To hold one byte of data in memory, you need 8 bits. However, as processors get faster and motherboards get more complicated it becomes necessary to be able to check that what you put into a memory location is the same as what you read from it. The PC has had this mechanism from day one, because in the early days of PCs motherboards and memory were less reliable than now, even though hardware speeds were much lower.
If you add a ninth bit to every byte of memory, you can store a 1 or 0 in that extra bit depending on whether the contents of the other 8 bits have an Even or Odd number of 1's. This provides a simple check to see that the contents of memory haven't been changed by stray electro-magnetic influences.
If you experience Memory Parity Errors, you should check that all memory (whether chips, SIMMs or SIPPs) modules are securely seated and they are all rated at the correct speed (or faster). Typical memory speeds are around 180 to 120 ns (nano-seconds) for XT machines and anywhere between 90 and 60 ns for more modern ATs.
Memory Hardware Types
Memory typically comes in three forms, chips (more properly known as DIPs - Dual Inline Packages), SIPPs (Single Inline Pin Package) and SIMMs (Single Inline Memory Module).
SIMMs are basically a replacement for SIPPs, both are small 2 x 9 cm PCBs with chips surface mounted onto them. There might be 8 or 9 chips (the 9 chip variety are those with parity and the most common), or the new breed of SIMMs with three chips, the two larger ones are 4 bits 'wide' each and the smaller one is for parity.
SIPPs have a row of pins at the base to plug into the motherboard or expansion card, SIMMs have a small edge connector, like expansion cards themselves.
SIMMs and SIPPs usually come in 256 Kb, 1Mb and now 4 Mb capacities.
DIPs can usually be identified by the last 4 digits of the part number on the chip casing. They take the form of XYYY, where X is the width in bits and YYY is the 'length' in kilo-bits. (Not all chip manufacturers use this notation.)
Thus if the code is 4256, then the chips are 4 bits wide and 256 kilo-bits (262144 bits) long, so you would need 2 to make 256 Kb because each is only half the width of one byte. You can simply say that each chip is 128 Kb in size. To make 1 Mb, you would need 8 chips, or 9 if parity is required.
Motherboards with more than 1 or 2 Mb fitted will usually have SIMM or SIPP sockets.
Static and Dynamic RAM
There are two types of RAM, that which has to be refreshed (read and rewritten) fairly often and that which does not. Dynamic RAM (the type that has to be refreshed) is cheap and therefore used for the majority of your PC's memory.
On XT machines you can actually alter the rate at which the timer circuitry refreshed the RAM, the less time spent refreshing the RAM, the more time the CPU / bus can be doing something else (an increase in PC speed of a few percent was possible). All you needed to do was slow down the rate of refresh rate until the machine stopped working, and then use a refresh rate just a tad faster than that! IBM stopped people mucking about like this with the introduction of the AT.
Static RAM does not need to be refreshed, because it holds its electrical charge better, this makes it much faster than dynamic RAM. It is also much more expensive, and is only used in critical areas of memory (see Processor Cache RAM below).
CMOS memory (Complimentary Metal-Oxide Semiconductor) can hold its charge for a very long length of time, while provided with a constant (but small) voltage. CMOS RAM (sometimes called Non-Volatile RAM or NVR) is used to hold the configuration information on AT machines and in some modems. Some CMOS chips require an external battery, while others have a small cell actually inside the chip casing. Both sorts of battery are usually trickle-charged while the machine or modem is powered from the mains.
Processor Cache RAM
Processor Cache RAM can be either internal (on the processor chip itself) or external. All 486s have an internal cache (8 Kb on Intel models, only 1 Kb on Cyrix's plug in 386sx replacement the 486SLC). Most 386s and 486s have external caches. This is to prevent the memory being a bottleneck in the system, as processor speeds start to outstrip the speed at which memory can store and retrieve information.
Processor RAM caches are always Static RAM rather than Dynamic RAM to give the speed advantage required.
Typical external cache sizes are 16, 32, 64, 128 or even 256 Kb. A cache of just a few Kb will allow the processor to fetch over 90% of instructions from the cache rather than main memory (called the hit-rate), after that the law of diminishing returns kicks in. The more cache you add over say 64 Kb the benefits you will see start to get smaller and smaller.
Hard Disk Caches
There are three types of Hard Disk Caches, internal, external and software. Most IDE, ESDI and SCSI drives will have a small read-ahead cache internally as part of the drive electronics of a few Kb. When the hard disk is asked for the data on a particular sector of the disk, the cache will buffer the next few sectors as well, as it is quite likely that they will be the next sectors asked for by the program.
External caches and software caches work in similar ways. External caches are usually situated on the drive controller card and populated with SIMMs, software caches are usually set up in Extended memory. Either kind can be a read-only or a read-write cache.
Read-only caches try to keep track of the data that is read from the disk most often as well as using read-ahead techniques similar to those described above. Areas such as the FAT (File Allocation Table) and directories of the disk are typical areas that are likely to be kept in the cache.
Hard Disk caches are useful because retrieving data from RAM is orders of magnitude faster than even the fastest hard drive.
Read-Write caches save up writes to the hard disk and tries to perform them in more efficient bursts, either when the processor doesn't have anything else to do, or to perform them in the most efficient order, meaning the drive heads don't have to travel so far.
One problem with read-write caches is that the power to the machine could be interrupted before crucial data is written to the hard disk. Most read-write caches trap the CTRL-ALT-DEL keypress, but none can prevent data loss through a power-cut or the accidental touch of the reset-button.
Top comments (1)
OMG, this brought back a lot of memories...
Menus in CONFIG.SYS and AUTOEXEC.BAT (clean boot, HIMEM.SYS, HIMEM+EMM, etc)
Every program/game needed a different configuration (frequent restarts).
(SET BLASTER=A220 I7 D1 and other options...)
Anyone heard of "unreal mode" ? (en.wikipedia.org/wiki/Unreal_mode)
I wrote a game (or base of a game) in unreal mode and then in WATCOM C++/DOS4GW
back in maybe 1995.
Those were heroic times...