I loved how so many companies tried different things like emulating an 8088 with the V20s. Commodore had their 128 which also included a Z80 capable of running C/PM. Sometimes it seems like we’ve lost that spirit of experimentation... at least on the hardware side of things.
Hi John. Completely agree! One of the great things about the original IBM PC was that there were lots of co-processor boards with things like 68000 or even Z8000 if you needed more computing power. IBM even made a PC that was capable of running mainframe software.
The BBC Micro also had the 'tube' that connected to a co-processor which of course allowed for them to quickly offer an early Arm based product.
Maybe today the answer would be to include an FPGA which can be reprogrammed with any core you want!
Thanks for the STM programming book link! MCU/embedded programming has been on my todo list for some time after mostly immersed in the x86 platform.
Have you seen MS-DOS demos aka "demoscene"? This is essentially bare metal programming on x86 with low level direct GPU programming before there were even such things as GPUs with the programmer having to implement his own rendering engine, translating 3D primitives to 2D primitves to VESA calls, bit blitting, etc and real time sound sythesis and/or with sampling. Later demos made use of the MIDI wavetable sound cards, which are now obsolete due to how powerful CPU software based synthesis has become. Heck, one could do bare metal assembly in DOS with the debug command. Back in the day there were a variety contests that also had interesting constraints such as size constraints e.g. 4k, 64k. Some interesting really tiny ones are even done in real-mode.
Hi! I'm aware of the demoscene and have watched a few videos. I find the idea fascinating but it's a bit hard to know where to start. Do you have any recommendations?
On topic of x86 low level programming have you see Justine Tunney's work on fitting LISP into a DOS boot sector (512 bytes) - https://github.com/jart/sectorlisp. Her blinkenlights x86 emulator is also really interesting - and potentially quite useful.
First Merry Christmas and no I haven't seen that LISP in-a-boot-sector code. That's incredible.
As far as writing demoscene code, I have no idea. I have some past familiarly with the high level graphical programming (X, OpenGL) but to do the entire pipeline down to rasterizer and video card driver level on bare metal is way beyond me. I suppose one can look at the DOOM code for the graphical implementation of a software rendering and game engine for something more complex that what's used in demos: https://github.com/id-Software/DOOM
For the music portion, all I know is that the "sound tracker" modules (X3M, MOD files) are the ancestral version of software wavetable synthesis. The first Unreal game and engine by Epic uses the same soundtracker system as these 90's era demos. Again implementing that is beyond me. I'm sure there's code out there but I don't know if any of the originals running on DOS have been open sourced.
As far as other bare metal programming on x86, if you consider that to include platform system , I/O, and hardware level programming I can recommend resources I'm familiar with. Note I have no association with the publishers. These are just books I've bought, and/or classes I took
They're the only one covering the topics of platform I/O and hardware standards and architecture as far as I'm aware, and are indispensable. I have a few of their books and took a couple classes in the past courtesy of my past employer. It allows one to understand the datasheets publicly available from Intel and AMD or those under NDA for driver development, as well as do low level analysis and hardware testing. I also have their The Unabridged Pentium 4: IA32 Processor Genealogy book, x86 Instruction Set Architecture, PCI System Architecture book, PCI Express 3.0 book, great resources, also available on Amazon for hefty prices.
I have the USB Complete: The Developer's Guide by Lakeview Research which I had to use breifly for a job but Mindshare also has their equivalent, which I don't have. I think they're the only ones however with the PCIE books and classes, and likewise CXL, Hypertransport, (AMD pre-Ryzen), NVME, etc
- caveat for x86 is that I think one still needs to know PCI / PCIE programming in order to memory map the registers in the fist place, since all on x86 platforms implementing PCI, they're behind a PCI-LPC/serial bridge, although one can opt for the more inefficient PIO method with INs and OUTs and polling, sometimes still needed for probing when there are no other methods of discovery, or on very limited platforms like Arduino
Hi, Happy New Year and apologies for the delayed response.
Thanks for such a detailed and comprehensive comment. There are lots and lots of links that I'll be following up on your list - particularly interested in the AVX and Virtualisation links.
As you may have seen I've added a paid tier, so as a small thank you for taking the time to comment I've added a complimentary one year membership to your subscription. Please let me know if this doesn't come through - I'm still getting familiar with paid subs!
Thank you very much! It's been great discovering my own tech roots, so to speak, that I took for granted or was largely ignorant of starting out as a kid and even now. It's amazing to see how things evolve, change and/or remain in use.
I loved how so many companies tried different things like emulating an 8088 with the V20s. Commodore had their 128 which also included a Z80 capable of running C/PM. Sometimes it seems like we’ve lost that spirit of experimentation... at least on the hardware side of things.
Hi John. Completely agree! One of the great things about the original IBM PC was that there were lots of co-processor boards with things like 68000 or even Z8000 if you needed more computing power. IBM even made a PC that was capable of running mainframe software.
The BBC Micro also had the 'tube' that connected to a co-processor which of course allowed for them to quickly offer an early Arm based product.
Maybe today the answer would be to include an FPGA which can be reprogrammed with any core you want!
Thanks for the STM programming book link! MCU/embedded programming has been on my todo list for some time after mostly immersed in the x86 platform.
Have you seen MS-DOS demos aka "demoscene"? This is essentially bare metal programming on x86 with low level direct GPU programming before there were even such things as GPUs with the programmer having to implement his own rendering engine, translating 3D primitives to 2D primitves to VESA calls, bit blitting, etc and real time sound sythesis and/or with sampling. Later demos made use of the MIDI wavetable sound cards, which are now obsolete due to how powerful CPU software based synthesis has become. Heck, one could do bare metal assembly in DOS with the debug command. Back in the day there were a variety contests that also had interesting constraints such as size constraints e.g. 4k, 64k. Some interesting really tiny ones are even done in real-mode.
Hi! I'm aware of the demoscene and have watched a few videos. I find the idea fascinating but it's a bit hard to know where to start. Do you have any recommendations?
On topic of x86 low level programming have you see Justine Tunney's work on fitting LISP into a DOS boot sector (512 bytes) - https://github.com/jart/sectorlisp. Her blinkenlights x86 emulator is also really interesting - and potentially quite useful.
First Merry Christmas and no I haven't seen that LISP in-a-boot-sector code. That's incredible.
As far as writing demoscene code, I have no idea. I have some past familiarly with the high level graphical programming (X, OpenGL) but to do the entire pipeline down to rasterizer and video card driver level on bare metal is way beyond me. I suppose one can look at the DOOM code for the graphical implementation of a software rendering and game engine for something more complex that what's used in demos: https://github.com/id-Software/DOOM
For the music portion, all I know is that the "sound tracker" modules (X3M, MOD files) are the ancestral version of software wavetable synthesis. The first Unreal game and engine by Epic uses the same soundtracker system as these 90's era demos. Again implementing that is beyond me. I'm sure there's code out there but I don't know if any of the originals running on DOS have been open sourced.
As far as other bare metal programming on x86, if you consider that to include platform system , I/O, and hardware level programming I can recommend resources I'm familiar with. Note I have no association with the publishers. These are just books I've bought, and/or classes I took
- First is the publisher and training company, MindShare: https://www.mindshare.com/
They're the only one covering the topics of platform I/O and hardware standards and architecture as far as I'm aware, and are indispensable. I have a few of their books and took a couple classes in the past courtesy of my past employer. It allows one to understand the datasheets publicly available from Intel and AMD or those under NDA for driver development, as well as do low level analysis and hardware testing. I also have their The Unabridged Pentium 4: IA32 Processor Genealogy book, x86 Instruction Set Architecture, PCI System Architecture book, PCI Express 3.0 book, great resources, also available on Amazon for hefty prices.
I have the USB Complete: The Developer's Guide by Lakeview Research which I had to use breifly for a job but Mindshare also has their equivalent, which I don't have. I think they're the only ones however with the PCIE books and classes, and likewise CXL, Hypertransport, (AMD pre-Ryzen), NVME, etc
- As for other resources:
No Starch:
https://nostarch.com/dive-systems
- Dive Into Systems
https://nostarch.com/art-64-bit-assembly-volume-1
- The Art of 64-Bit Assembly, Volume 1
https://nostarch.com/book-i%C2%B2c
- The Book of I2C
- caveat for x86 is that I think one still needs to know PCI / PCIE programming in order to memory map the registers in the fist place, since all on x86 platforms implementing PCI, they're behind a PCI-LPC/serial bridge, although one can opt for the more inefficient PIO method with INs and OUTs and polling, sometimes still needed for probing when there are no other methods of discovery, or on very limited platforms like Arduino
Apress (Springer):
https://link.springer.com/book/10.1007/978-1-4842-5890-3
- Hands-on Booting: Learn the Boot Process of Linux, Windows, and Unix
https://link.springer.com/book/10.1007/978-1-4842-7939-7
- System Firmware: An Essential Guide to Open Source and Embedded Solutions (2022)
- covers BIOS, UEFI, coreboot, etc.
Older predecessor of the book by one of the authors, available for free in pdf and epub:
https://link.springer.com/book/10.1007/978-1-4842-0070-4
- Embedded Firmware Solutions (2015)
https://link.springer.com/book/10.1007/978-1-4842-6106-4#otherversion=9781484261064
- Building Secure Firmware
https://link.springer.com/book/10.1007/978-1-4302-6584-9
- A Practical Guide to TPM 2.0
- available for free in pdf and epub
https://link.springer.com/book/10.1007/978-1-4842-2403-8
- Low-Level Programming: C, Assembly, and Program Execution on Intel® 64 Architecture
- repo: https://github.com/Apress/low-level-programming
https://link.springer.com/book/10.1007/978-1-4842-4063-2
- Modern X86 Assembly Language Programming, 2nd Ed: Covers x86 64-bit, AVX, AVX2, and AVX-512
- repo: https://github.com/Apress/modern-x86-assembly-language-programming-2e
By the same author:
https://link.springer.com/book/10.1007/978-1-4842-7918-2
- Modern Parallel Programming with C++ and Assembly Language: X86 SIMD Development Using AVX, AVX2, and AVX-512
- repo: https://github.com/Apress/modern-parallel-programming-cpp-assembly
https://link.springer.com/book/10.1007/978-1-4842-5076-1
- Beginning x64 Assembly Programming: From Novice to AVX Professional
- repo: https://github.com/Apress/beginning-x64-assembly-programming
Packt:
https://www.packtpub.com/product/modern-computer-architecture-and-organization/9781838984397
- Modern Computer Architecture and Organization
- starts the intro with Charles Babbage's Analytical Engine, hah!
- more broad architectures overview that touches on details where needed than a specific programming guide
- https://github.com/packtpublishing/modern-computer-architecture-and-organization
https://www.packtpub.com/product/mastering-assembly-programming/9781787287488
- Mastering Assembly Programming
- repo: https://github.com/PacktPublishing/Mastering-Assembly-Programming
https://www.packtpub.com/product/linux-kernel-programming/9781789953435
- Linux Kernel Programming (Part 1)
- repo: https://github.com/PacktPublishing/Linux-Kernel-Programming
https://www.packtpub.com/free-ebook/linux-kernel-programming-part-2-char-device-drivers-and-kernel-synchronization/9781801079518
- Linux Kernel Programming Part 2 - Char Device Drivers and Kernel Synchronization
- repo: https://github.com/PacktPublishing/Linux-Kernel-Programming-Part-2
https://www.packtpub.com/product/mastering-linux-device-driver-development/9781789342048
- Mastering Linux Device Driver Development
https://www.packtpub.com/product/mastering-embedded-linux-programming-third-edition/9781789530384
- Mastering Embedded Linux Programming - Third Edition
- repo: https://github.com/PacktPublishing/Mastering-Embedded-Linux-Programming-Third-Edition
https://www.packtpub.com/product/mastering-kvm-virtualization-second-edition/9781838828714
- Mastering KVM Virtualization - Second Edition
- not at the bare metal level but above that as kernel and system programming level
- for bare metal / h/w programming see Mindshare:
https://www.mindshare.com/Learn/PC_Virtualization/Training
- PC Virtualization e.g. VT-x
https://www.mindshare.com/Learn/IO_Virtualization_on_Intel_Platforms
- IO Virtualization on Intel Platforms
- see also:
https://software.intel.com/content/www/us/en/develop/download/intel-virtualization-technology-for-directed-io-architecture-specification.html
- Intel ® Virtualization Technology for Directed I/O (VT-d)
- necessary for programming and debugging the IOMMU on Intel platforms
Hi, Happy New Year and apologies for the delayed response.
Thanks for such a detailed and comprehensive comment. There are lots and lots of links that I'll be following up on your list - particularly interested in the AVX and Virtualisation links.
As you may have seen I've added a paid tier, so as a small thank you for taking the time to comment I've added a complimentary one year membership to your subscription. Please let me know if this doesn't come through - I'm still getting familiar with paid subs!
Thanks again!
Thank you very much! It's been great discovering my own tech roots, so to speak, that I took for granted or was largely ignorant of starting out as a kid and even now. It's amazing to see how things evolve, change and/or remain in use.
Great edition. I am going to add Feynman Lectures on Computation to my reading list. 🙏
Hi Nick. Thanks so much. If you'd like to browse the Feynman then I believe that there is a copy on the Internet Archive!