diff --git a/COPYING b/COPYING deleted file mode 100644 index 652426c4..00000000 --- a/COPYING +++ /dev/null @@ -1,19 +0,0 @@ -Copyright © 2007-2009 David Given - -Permission is hereby granted, free of charge, to any person obtaining a copy -of this software and associated documentation files (the "Software"), to deal -in the Software without restriction, including without limitation the rights -to use, copy, modify, merge, publish, distribute, sublicense, and/or sell -copies of the Software, and to permit persons to whom the Software is -furnished to do so, subject to the following conditions: - -The above copyright notice and this permission notice shall be included in -all copies or substantial portions of the Software. - -THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR -IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, -FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE -AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER -LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, -OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN -THE SOFTWARE. \ No newline at end of file diff --git a/COPYING.md b/COPYING.md new file mode 100644 index 00000000..f0ea080b --- /dev/null +++ b/COPYING.md @@ -0,0 +1,19 @@ +Copyright © 2018-2019 David Given + + - Permission is hereby granted, free of charge, to any person obtaining a copy + of this software and associated documentation files (the "Software"), to + deal in the Software without restriction, including without limitation + the rights to use, copy, modify, merge, publish, distribute, sublicense, + and/or sell copies of the Software, and to permit persons to whom the + Software is furnished to do so, subject to the following conditions: + + The above copyright notice and this permission notice shall be included + in all copies or substantial portions of the Software. + + - THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS + OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF + MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN + NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, + DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR + OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE + USE OR OTHER DEALINGS IN THE SOFTWARE. \ No newline at end of file diff --git a/LICENSE b/LICENSE deleted file mode 100644 index f17727ed..00000000 --- a/LICENSE +++ /dev/null @@ -1,21 +0,0 @@ -MIT License - -Copyright (c) 2018 David Given - -Permission is hereby granted, free of charge, to any person obtaining a copy -of this software and associated documentation files (the "Software"), to deal -in the Software without restriction, including without limitation the rights -to use, copy, modify, merge, publish, distribute, sublicense, and/or sell -copies of the Software, and to permit persons to whom the Software is -furnished to do so, subject to the following conditions: - -The above copyright notice and this permission notice shall be included in all -copies or substantial portions of the Software. - -THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR -IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, -FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE -AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER -LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, -OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE -SOFTWARE. diff --git a/README.md b/README.md index f98f319d..76badddf 100644 --- a/README.md +++ b/README.md @@ -1,6 +1,9 @@ FluxEngine ========== +(If you're reading this on GitHub, the formatting's a bit messed up. [Try the +version on cowlark.com instead.](http://cowlark.com/fluxengine/) + What? ----- @@ -10,78 +13,56 @@ PC drive to accept Amiga disks, CLV Macintosh disks, bizarre 128-sector CP/M disks, and other weird and bizarre formats. (Although not all of these are supported yet. I could really use samples.) -a FluxEngine attached to a floppy drive +The hardware consists of a single, commodity part with a floppy drive +connector soldered onto it. No ordering custom boards, no fiddly surface +mount assembly, and no fuss: nineteen simpler solder joints and you're done. + +Don't believe me? Watch the demo reel! + +
+ +
**Important note.** On 2019-02-09 I did a hardware redesign and moved the pins on the board. Sorry for the inconvenience, but it means you don't have to modify the board any more to make it work. If you built the hardware prior to then, you'll need to adjust it. -**New!** There's a demo reel! +Where? +------ -
- -
+It's [open source on GitHub!](https://github.com/davidgiven/fluxengine) -### Infrequently asked questions because nobody's got round to asking them yet +How? +---- -**Q.** Why not just use a USB floppy drive? There are lots and they're cheap. +This page was getting kinda unwieldy so I've broken it up. Please consult the +following friendly articles: -**A.** Because USB floppy drives typically support a very limited set of -formats --- typically only IBM 1440kB and 720kB. The FluxEngine should work -on (almost) anything. + - [Frequently asked questions](doc/faq.md) ∾ but why...? ∾ does it...? ∾ can it...? + + - [How the FluxEngine works](doc/technical.md) ∾ nitty gritty of the + sampler/sequencer hardware ∾ useful links on floppy drives ∾ why I'm not + using an Arduino/STM32/ESP32/Raspberry Pi -**Q.** But aren't floppy disks obsolete? + - [Making a FluxEngine](doc/building.md) ∾ what parts you need ∾ building it ∾ + setting up the toolchain ∾ compiling the firmware ∾ programming the board -**A.** Absolutely they are. That doesn't mean they've gone away. Good luck -with any old hardware, for example; a classic Mac won't boot without a -classic Mac boot disk, and you can't make them on PCs (because they're -weird). This is where the FluxEngine comes in. + - [Using a FluxEngine](doc/using.md) ∾ what to do with your new hardware ∾ + flux files and image files ∾ knowing what you're doing -**Q.** But how can you read and write non-PC formats on a PC floppy drive? + - [Reading dubious disks](doc/problems.md) ∾ it's not an exact science ∾ + the sector map ∾ clock detection and the histogram ∾ tuning the clock ∾ + manual adjustment -**A.** Because the FluxEngine hardware simply streams the raw magnetic flux -pulsetrain from the drive to the PC, and then the analysis is done off-line -in software. It doesn't rely on any floppy disk controller to interpret the -pulsetrain, so we can be a lot cleverer. In fact, the disk doesn't even have -to be spinning at the same speed. +Supported disk formats +---------------------- -**Q.** Does it work on 5.25" drives? +The current support state is as follows. -**A.** Yes! Although PC 5.25" drives spin at 360 RPM rather than 300 RPM, -which means there's only 166ms of data on one per track rather than 200ms; -if you try to write a 3.5" format disk onto one it probably won't work. - -**Q.** Does it work on 8" drives? - -**A.** Probably? You'd need an adapter to let you connect the drive to the -FluxEngine --- [you can get them](http://www.dbit.com/fdadap.html). I don't -have either the adapter, the drive, or any 8" disks. If anyone wants to give -it a try, please [tell me about -it](https://github.com/davidgiven/fluxengine/issues/new). - -**Q.** Is this like KryoFlux? Do you support KryoFlux stream files? - -**A.** It's very like KryoFlux, although much simpler. Yes, FluxEngine can -read from KryoFlux stream files (but not write to them yet; nobody's asked). -FluxEngine doesn't capture all the data that KryoFlux does, like index -markers. - -**Q.** I've tried it and my disk doesn't work! - -**A.** [I have an entire page on diagnosing read failures.](doc/problems.md) - -**Q.** That's awesome! What formats does it support? - -**A.** I'm glad you asked the question. Consult the following table! - -### Formats that it supports - -Here's the table. - -| Format | Read? | Write? | Notes | -|:---------------------------------------|:-----:|:------:|-------| -| IBM PC compatible | 🦄 | | and compatibles (like the Atari ST) | +| Format | Read? | Write? | Notes | +|:-----------------------------------------|:-----:|:------:|-------| +| IBM PC compatible | 🦄 | | and compatibles (like the Atari ST) | | [Acorn ADFS](doc/disk-acornadfs.md) | 🦄 | | single- and double- sided | | [Acorn DFS](doc/disk-acorndfs.md) | 🦄 | | | | [AES Superplus / No Problem](doc/disk-aeslanier.md) | 🦖 | | hard sectors! and _very_ experimental | @@ -132,240 +113,6 @@ most likely be able to read the data fine, unless they're doing bizarre things like spiral tracks or partially encoded data, but let's stick with normal conventionally formatted disks for the time being! -### Big list of stuff to work on - -Both the firmware and the software are works in progress. There's still things -to do. - - - Support for more floppy disk formats! I'm trying to scare up some sample - disks to image and decode; I'm particularly looking for Commodore 64 1541 - disks and Apple Macintosh 800kB CLV disks. If you have any which I can - borrow and you live in (or near) Switzerland, please [get in - touch](https://github.com/davidgiven/fluxengine/issues/new). - - - Better (and more) write support. The hardware can write disks, but not much - of the software's done. Writing is hard because I really need the device - to test the disks on. I could use some help here. Also, most of the write - code is a bit of a disaster and needs a major refactoring. - - - Sourcing a different microcontroller. The PSoC5 is a great thing to work - with, but it's disappointingly proprietary and the toolchain only works on - Windows. It'd be nice to support something easier to work with. I need a 5V - microcontroller (which unfortunately rules out Arduinos) with at least - seventeen GPIO pins in a row. As non-PSoC5 microcontrollers won't have the - FPGA soft logic, that also means I'd most likely need to bitbang the floppy - drive, so speed is important. If you have any recommendations, please [get - in touch](https://github.com/davidgiven/fluxengine/issues/new). - -Where? ------- - -It's [open source on GitHub!](https://github.com/davidgiven/fluxengine) - -How? ----- - -### Introduction - -The system is based around an off-the-shelf Cypress development board, -costing about $10 plus shipping. The only hardware modification you need to -do is to attach a floppy drive connector. - -[I wrote a long, detailed set of instructions on building and programming -it.](doc/building.md) - -I'm going to assume you've done this. - -### Using it - - 1. Attach the FluxEngine to your floppy disk cable. - - If you built your board using a connector, this is easy: just plug it in. If you're using pins, you need to make sure that the red strip on the cable is **to the right** (next to pin 2.7 on the board), and that the pins plug into the **lower set of holes** in the connector (so the connector overhangs the top of the board). - - 2. Attach the cable to your drives. - - Floppy disk cables typically have [two pairs of floppy disk drive - connectors with a twist between - them](http://www.nullmodem.com/Floppy.htm). (Each pair has one connector - for a 3.5" drive and a different one for a 5.25" drive.) Normally the - first floppy disk drive is drive B, and the one after the twist is drive - A. However, FluxEngine considers the first drive to be drive 0 and the - second one to be drive 1. It's this way so as to allow the FluxEngine to - be plugged directly onto the back of a disk drive. If you only have one - drive, remember to plug the drive into the connector _before_ the twist. - (Or use `-s :d=1` to select drive 1 when working with disks.) - - 2. **Important.** Make sure that no disk you care about is in the drive. - (Because if your wiring is wrong and a disk is inserted, you'll probably - corrupt it.) - - 3. Connect the floppy drive to power. Nothing should happen. If anything - does, disconnect it and check step 1. - - 4. Connect the FluxEngine to your PC via USB --- using the little socket on - the board, not the big programmer plug. - - 5. Do `.obj/fe-rpm` from the shell. The motor should work and it'll tell you - that the disk is spinning at about 300 rpm. If it doesn't, please [get - in touch](https://github.com/davidgiven/fluxengine/issues/new). - - 6. Insert a standard PC formatted floppy disk into the drive (probably a good - idea to remove the old disk first). Then do `.obj/fe-readibm`. It should - read the disk, emitting copious diagnostics, and spit out an `ibm.img` - file containing the decoded disk image (either 1440kB or 720kB depending). - - 7. Success! - -### The programs - -#### Source and destination specifiers - -When reading from or writing to _a disk_ (or a file pretending to be a disk), -use the `-s` and `-d` options to tell FluxEngine which bits of the disk you -want to access. These use a common syntax: - -``` -fe-readibm -s fakedisk.flux:t=0-79:s=0 -``` - - - To access a real disk, leave out the filename (so `:t=0-79:s=0`). - - - To access only some tracks, use the `t=` modifier. To access only some - sides, use the `s=` modifier. To change drives, use `d=`. - - - Inside a modifier, you can use a comma separated list of ranges. So - `:t=0-3` and `:t=0,1,2,3` are equivalent. - - - When specifying a range, you can also specify the step. For example, - `:t=0-79x2` would be used when accessing a 40-track disk with double - stepping. - - - To read from drive 1 instead of drive 0, use `:d=1`. - - - To read from a set of KryoFlux stream files, specify the path to the - directory containing the files _with a trailing slash_; so - `some/files/:t=0-10`. There must be a files for a single disk only - in the directory. - -Source and destination specifiers work entirely in *physical units*. -FluxEngine is intended to be connected to an 80 (or 82) track double sided -drive, and these are the units used. If the format you're trying to access -lays out its tracks differently, then you'll need a specifier which tells -FluxEngine how to find those tracks. See the 40-track disk example above. - -If you _don't_ specify a modifier, you'll get the default, which should be -sensible for the command you're using. - -**Important note:** FluxEngine _always_ uses zero-based units (even if the -*disk format says otherwise). - -### The commands - -The FluxEngine client software is a largely undocumented set of small tools. -You'll have to play with them. They all support `--help`. They're not -installed anywhere and after building you'll find them in the `.obj` -directory. - - - `fe-erase`: wipes (all or part of) a disk --- erases it without writing - a pulsetrain. - - - `fe-inspect`: dumps the raw pulsetrain / bitstream to stdout. Mainly useful - for debugging. - - - `fe-read*`: reads various formats of disk. See the per-format documentation - linked from the table above. - - - `fe-write*`: writes various formats of disk. Again, see the per-format - documentation above. - - - `fe-writeflux`: writes raw flux files. This is much less useful than you - might think: you can't necessarily copy flux files read from a disk, - because errors in the sampling are compounded and the result probably - isn't readable. It's mainly useful for flux files synthesised by the - other `fe-write*` commands. - - - `fe-writetestpattern`: writes regular pulses (at a configurable interval) - to the disk. Useful for testing drive jitter, erasing disks in a more - secure fashion, or simply debugging. Goes well with `fe-inspect`. - - - `fe-readibm`: reads 720kB or 1440kB IBM MFM disks. Emits a standard - filesystem image. - - - `fe-rpm`: measures the RPM of the drive (requires a disk in the drive). - Mainly useful for testing. - - - `fe-seek`: moves the head. Mainly useful for finding out whether your drive - can seek to track 82. (Mine can't.) - - - `fe-testbulktransport`: measures your USB throughput. You need about 600kB/s - for FluxEngine to work. You don't need a disk in the drive for this one. - - - `fe-upgradefluxfile`: occasionally I need to upgrade the flux file format in - a non-backwards-compatible way; this tool will upgrade flux files to the new - format. - -Commands which take `--source` or `--dest` take a parameter of the syntax -`$FILENAME:$MODIFIER:$MODIFER...` as described above. If left unspecified, -you get the default specified by the command, which will vary depending on -which disk format you're using (and is usually the right one). - -### How it works - -It's very very simple. The firmware measures the time between flux transition -pulses, encodes them, and streams them over USB to the PC. - -There's an 8-bit counter attached to an 12MHz clock. This is used to measure -the interval between pulses. If the timer overflows, we pretend it's a pulse -(this very rarely happens in real life). ([I'm working on -this.](https://github.com/davidgiven/fluxengine/issues/8)) - -An HD floppy has a nominal clock of 500kHz, so we use a sample clock of 12MHz -(every 83ns). This means that our 500kHz pulses will have an interval of 24 -(and a DD disk with a 250kHz nominal clock has an interval of 48). This gives -us more than enough resolution. If no pulse comes in, then we sample on -rollover at 21us. - -(The clock needs to be absolutely rock solid or we get jitter which makes the -data difficult to analyse, so 12 was chosen to be derivable from the -ultra-accurate USB clock.) - -Once at the PC, we do some dubious heuristics to determine the clock rate, -which depends on what kind of encoding is being used, and that in turn lets -us decode the pulsetrain into actual bits and derive the raw floppy disk -records from there. Reassembling the image and doing stuff like CRC checking -is straightforward. - -Some useful and/or interesting numbers: - - - nominal rotation speed is 300 rpm, or 5Hz. The period is 200ms. - - a pulse is 150ns to 800ns long. - - a 12MHz tick is 83ns. - - MFM HD encoding uses a clock of 500kHz. This makes each recording cell 2us, - or 24 ticks. For DD it's 4us and 48 ticks. - - a short transition is one cell (2us == 24 ticks). A medium is a cell and - a half (3us == 36 ticks). A long is two cells (4us == 48 ticks). Double - that for DD. - - pulses are detected with +/- 350ns error for HD and 700ns for DD. That's - 4 ticks and 8 ticks. That seems to be about what we're seeing. - - in real life, start going astray after about 128 ticks == 10us. If you - don't write anything, you read back random noise. - -Useful links: - - - [The floppy disk user's - guide](http://www.hermannseib.com/documents/floppy.pdf): an incredibly - useful compendium of somewhat old floppy disk information --- which is - fine, because floppy disks are somewhat old. - - - [The TEAC FD-05HF-8830 data - sheet](https://hxc2001.com/download/datasheet/floppy/thirdparty/Teac/TEAC%20FD-05HF-8830.pdf): - the technical data sheet for a representative drive. Lots of useful - timing numbers here. - - - [KryoFlux stream file - documentation](https://www.kryoflux.com/download/kryoflux_stream_protocol_rev1.1.pdf): - the format of KryoFlux stream files (partially supported by FluxEngine) - Who? ---- @@ -376,6 +123,13 @@ There may or may not be anything interesting there. License ------- -Everything here is licensed under the MIT open source license. Please see +Everything here _except the contents of the `dep` directory_ is © 2019 David +Given and is licensed under the MIT open source license. Please see [COPYING](COPYING) for the full text. The tl;dr is: you can do what you like with it provided you don't claim you wrote it. + +As an exception, `dep/fmt` contains a copy of [fmt](http://fmtlib.net), +maintained by Victor Zverovich (`vitaut `_) and +Jonathan Müller (`foonathan `_) with +contributions from many other people. It is licensed under the terms of the +BSD license. Please see the contents of the directory for the full text. \ No newline at end of file diff --git a/doc/faq.md b/doc/faq.md new file mode 100644 index 00000000..0df7af46 --- /dev/null +++ b/doc/faq.md @@ -0,0 +1,59 @@ +Frequently asked questions +========================== + +**Q.** Why not just use a USB floppy drive? There are lots and they're cheap. + +**A.** Because USB floppy drives typically support a very limited set of +formats --- typically only IBM 1440kB and 720kB. The FluxEngine should work +on (almost) anything, including the ones that IBM machines won't touch. Also, +as it's USB, it'll work happily on machines that were never designed for +floppy disks (like my development Chromebook). + +**Q.** But aren't floppy disks obsolete? + +**A.** Absolutely they are. That doesn't mean they've gone away. Good luck +with any old hardware, for example; a classic Mac won't boot without a +classic Mac boot disk, and you can't make them on PCs (because they're +weird). This is where the FluxEngine comes in. + +**Q.** But how can you read and write non-PC formats on a PC floppy drive? + +**A.** Because the FluxEngine hardware simply streams the raw magnetic flux +pulsetrain from the drive to the PC, and then the analysis is done off-line +in software. It doesn't rely on any floppy disk controller to interpret the +pulsetrain, so we can be a lot cleverer. In fact, the disk doesn't even have +to be spinning at the same speed. + +**Q.** Does it work on 5.25" drives? + +**A.** Yes! Although PC 5.25" drives spin at 360 RPM rather than 300 RPM, +which means there's only 166ms of data on one per track rather than 200ms; +if you try to write a 3.5" format disk onto one it probably won't work. + +**Q.** Does it work on 8" drives? + +**A.** Probably? You'd need an adapter to let you connect the drive to the +FluxEngine --- [you can get them](http://www.dbit.com/fdadap.html). I don't +have either the adapter, the drive, or any 8" disks. If anyone wants to give +it a try, please [tell me about +it](https://github.com/davidgiven/fluxengine/issues/new). + +**Q.** Is this like KryoFlux / Catweasel / DiskFerret? Do you support KryoFlux stream files? + +**A.** It's very like all of these; the idea's old, and lots of people have +*tried it (you can get away with any sufficiently fast microcontroller and +*enough RAM). FluxEngine can +read from KryoFlux stream files natively, and there's a tool which will let +you convert at least one kind of Catweasel file to FluxEngine's native flux +file format. + +**Q.** Can I use this to make exact copies of disks? + +**A.** No. FluxEngine can read disks, and it can write disks, but it can't +write flux files that it read. There's several reasons for this, including +but not limited to: amplification of disk noise causing unreadable disks; +needing to rearrange the read data so it all fits exactly between the index +markers; reproduction of non-repeatable noise (this was a common trick with +Apple II copy protection); etc. What FluxEngine prefers is to read a disk, +turn it into a filesystem image, and then synthesise flux from the filesystem +image and write _that_ to another disk. diff --git a/doc/floppy.jpg b/doc/floppy.jpg deleted file mode 100644 index 2da45fe5..00000000 Binary files a/doc/floppy.jpg and /dev/null differ diff --git a/doc/technical.md b/doc/technical.md new file mode 100644 index 00000000..f4e48629 --- /dev/null +++ b/doc/technical.md @@ -0,0 +1,123 @@ +How the FluxEngine works +======================== + +## The hardware design + +It's essentially very simple, although the details are complicated. In +essence, all the firmware does is to measure the time between flux transition +pulses and send them to the PC. + +There's an 8-bit counter attached to an 12MHz clock. This is used to measure +the interval between pulses. A Cypress datapath state machine thingy using +Verilog turns the timer, pulse and index information into a bytecode stream +which encodes intervals, pulses, and whether the index hole has been seen. + +This is then streamed back to the PC, where offline software decodes it: it +does simple statistical analysis to guess the clock rate, then turns the +pulses into a nice, lined up bit array and from there decodes the file system +format. + +Writing back to disk works the same way: bytes are streamed to the +FluxEngine, where a different datapath state machine thingy (the PSoC5LP has +24, all independently programmable) to interpret the bytecodes, generate a +stream of pulses to the disk. + +The bytecode format represents an interval between pulses as a byte, a pulse +as a byte, and the index hole as a byte. Timer overflows are handled by +sending multiple intervals in a row. + +An HD floppy has a nominal pulse frequency of 500kHz, and we use a sample +clock of 12MHz (every 83ns). This means that our 500kHz pulses will have an +average interval of 24. This gives us more than enough resolution. At this +speed, in the 200ms that a 3.5" disk takes to rotate, we will see about +100,000 pulses. Each one is encoded as two bytes, one for the interval and +one to generate the pulse; so that revolution generates 200kB of data. +(Extremely approximately. The actual figure is less.) + +(The clock needs to be absolutely rock solid or we get jitter which makes the +data difficult to analyse, so 12 was chosen to be derivable from the +ultra-accurate USB clock.) + +Some useful and/or interesting numbers: + + - nominal rotation speed is 300 rpm, or 5Hz. The period is 200ms. + - a pulse, as generated by the floppy drive electronics, is 150ns to 800ns + long. + - a 12MHz tick is 83ns. + - MFM HD encoding uses a clock of 500kHz. This makes each recording cell 2us, + or 24 ticks. For DD it's 4us and 48 ticks. + - a short transition is one cell (2us == 24 ticks). A medium is a cell and + a half (3us == 36 ticks). A long is two cells (4us == 48 ticks). Double + that for DD. + - pulses are detected with +/- 350ns error for HD and 700ns for DD. That's + 4 ticks and 8 ticks. That seems to be about what we're seeing. + - in real life, pulses start going astray on a 3.5" drive after about 128 + ticks == 10us. I haven't tried with a 5.25" drive yet as I don't have a + 5.25" scratch disk. + +## Why don't I use an Arduino / STM32 / ESP32 / Raspberry Pi / etc? + +I've got a _lot_ of questions on this, and multiple Github issues of people +debating it. It's complicated, but it's essentially a tradeoff between speed +and complexity. + +FluxEngine's read process involves generating a lot of data using a fairly +brute force sampling approach --- about 150kB per disk revolution, and +sometimes it needs to record multiple revolutions. Most microcontrollers +don't have enough RAM to buffer this, so instead I have to stream it over USB +back to the host PC in real time. The disk won't wait, so I need to stream data faster +than the disk is producing it: the total is about 800kB/s. + +Handling USB is pretty CPU-hungry, so my candidate microntroller has to be +able to cope with the ruinously strict real-time requirements of the +sampler's 12MHz clock as well as keeping up with 13,000 USB interrupts a +second (one for each 64-byte frame) in order to transfer the data. + +The Atmels and STM32s I found were perfectly capable of doing the real-time +sampling, using hand-tool assembly, but I very much doubt whether they could +do the USB streaming as well (although I'd like to move away from the Cypress +onto something less proprietary and easier to source, so I'd like to be +proven wrong here). + +The Raspberry Pi easily has enough processing power and memory, but it's also +got terrible GPIO pin read performance --- [about +1kHz](https://raspberrypi.stackexchange.com/questions/9646/how-fast-is-gpiodma-multi-i2s-input/10197#10197). +That's a long way from the 12MHz I need. + +The PSoC5LP part I'm using has enough CPU to handle the USB side of things, +and it _also_ has a whole set of FPGA-like soft programmable features, +including 24 mini-ALU systems that are ideally suited to exactly this kind of +sampling. I can read the disk and generate the byte stream describing the +flux pattern entirely in 'hardware', without involving the main CPU at all. +This is then DMAed directly into a set of ring buffers read for the USB +system to pick up and relay back to the PC. It's incredibly simple and works +well. (The same applies to writing flux back onto the disk.) + +The development board I'm using, the +[CY8CKIT-059](https://www.cypress.com/documentation/development-kitsboards/cy8ckit-059-psoc-5lp-prototyping-kit-onboard-programmer-and), +also has another big advantage: it's the right shape. It's got 17 holes in a +row connected to GPIO pins which means I can just connect a floppy drive +connector directly to the board without needing to build any hardware. No +adapter board, no special cable, nothing. This makes the FluxEngine hardware +incredibly easy to assemble, which therefore means cheap. + +Speaking of which, the CY8CKIT-059 is $10. (Before shipping, which is +admittedly expensive.) + +### Some useful links + + - [The floppy disk user's + guide](http://www.hermannseib.com/documents/floppy.pdf): an incredibly + useful compendium of somewhat old floppy disk information --- which is + fine, because floppy disks are somewhat old. + + - [The TEAC FD-05HF-8830 data + sheet](https://hxc2001.com/download/datasheet/floppy/thirdparty/Teac/TEAC%20FD-05HF-8830.pdf): + the technical data sheet for a representative drive. Lots of useful + timing numbers here. + + - [KryoFlux stream file + documentation](https://www.kryoflux.com/download/kryoflux_stream_protocol_rev1.1.pdf): + the format of KryoFlux stream files (partially supported by FluxEngine) + + \ No newline at end of file diff --git a/doc/using.md b/doc/using.md new file mode 100644 index 00000000..8ae842f7 --- /dev/null +++ b/doc/using.md @@ -0,0 +1,190 @@ +Using a FluxEngine +================== + +So you've [built the hardware](building.md)! What now? + +## Connecting it up + +In order to do anything useful, you have to plug it in to a floppy disk drive (or two). + + 1. Plug the motherboard end of your floppy disk cable into the FluxEngine. + + The **red stripe goes on the right**. The **lower set of + holes connect to the board**. (Pin 2 of the connector needs to connect + to pin 2.7 on the board.) + + If you're using header pins, the upper row of holes in the connector + should overhang the edge of the board. If you're using a floppy drive + motherboard connector, you're golden, of course (unless you have one of + those annoying unkeyed cables, or have accidentally soldered the + connector on in the wrong place --- don't laugh, I've done it.) + + 2. Plug the drive end of your floppy disk cable into the drive (or drives). + + Floppy disk cables typically have [two pairs of floppy disk drive + connectors with a twist between + them](http://www.nullmodem.com/Floppy.htm). (Each pair has one connector + for a 3.5" drive and a different one for a 5.25" drive.) (Some cables + are cheap and just have the 3.5" connectors. Some are _very_ cheap and + have a single 3.5" connector, after the twist.) + + FluxEngine uses, sadly, non-standard disk numbering (there are reasons). + Drive 0 is the one nearest the motherboard; that is, before the twist. + Drive 1 is the one at the end of the cable; that is, after the twist. + Drive 0 is the default. If you only have one drive, remember to plug the + drive into the connector _before_ the twist. (Or use `-s :d=1` to select + drive 1 when working with disks.) + + 3. **Important.** Make sure that no disk you care about is in the drive. + (Because if your wiring is wrong and a disk is inserted, you'll corrupt it.) + + 4. Connect the floppy drive to power. Nothing should happen. If you've + connected something in backwards, you'll see the drive light up, the + motor start, and if you didn't take the disk out, one track has just + been wiped. If this happens, check your wiring. + + 5. Connect the FluxEngine to your PC via USB --- using the little socket on + the board, not the big programmer plug. + + 6. Insert a scratch disk and do `.obj/fe-rpm` from the shell. The motor + should work and it'll tell you that the disk is spinning at about 300 + rpm for a 3.5" disk, or 360 rpm for a 5.25" disk. If it doesn't, please + [get in touch](https://github.com/davidgiven/fluxengine/issues/new). + + 7. Do `.obj/fe-testbulktransport` from the shell. It'll measure your USB + bandwidth. Ideally you should be getting above 900kB/s. FluxEngine needs + about 850kB/s, so if you're getting less than this, try a different USB + port. + + 8. Insert a standard PC formatted floppy disk into the drive (probably a good + idea to remove the old disk first). Then do `.obj/fe-readibm`. It should + read the disk, emitting copious diagnostics, and spit out an `ibm.img` + file containing the decoded disk image (either 1440kB or 720kB depending). + + 9. Profit! + +## The programs + +I'm sorry to say that the programs are very badly documented --- they're +moving too quickly for the documentation to keep up. They do all respond to +`--help`. There are some common properties, described below. + +### Source and destination specifiers + +When reading from or writing to _a disk_ (or a file pretending to be a disk), +use the `--source` (`-s`) and `--dest` (`-d`) options to tell FluxEngine +which bits of the disk you want to access. These use a common syntax: + +``` +.obj/fe-readibm -s fakedisk.flux:t=0-79:s=0 +``` + + - To access a real disk, leave out the filename (so `:t=0-79:s=0`). + + - To access only some tracks, use the `t=` modifier. To access only some + sides, use the `s=` modifier. To change drives, use `d=`. + + - Inside a modifier, you can use a comma separated list of ranges. So + `:t=0-3` and `:t=0,1,2,3` are equivalent. + + - When specifying a range, you can also specify the step. For example, + `:t=0-79x2` would be used when accessing a 40-track disk with double + stepping. + + - To read from drive 1 instead of drive 0, use `:d=1`. + + - To read from a set of KryoFlux stream files, specify the path to the + directory containing the files _with a trailing slash_; so + `some/files/:t=0-10`. There must be a files for a single disk only + in the directory. + +Source and destination specifiers work entirely in *physical units*. +FluxEngine is intended to be connected to an 80 (or 82) track double sided +drive, and these are the units used. If the format you're trying to access +lays out its tracks differently, then you'll need a specifier which tells +FluxEngine how to find those tracks. See the 40-track disk example above. + +If you _don't_ specify a modifier, you'll get the default, which should be +sensible for the command you're using. + +**Important note:** FluxEngine _always_ uses zero-based units (even if the +*disk format says otherwise). + +### The commands + +The FluxEngine client software is a largely undocumented set of small tools. +You'll have to play with them. They all support `--help`. They're not +installed anywhere and after building you'll find them in the `.obj` +directory. + + - `fe-erase`: wipes (all or part of) a disk --- erases it without writing + a pulsetrain. + + - `fe-inspect`: dumps the raw pulsetrain / bitstream to stdout. Mainly useful + for debugging. + + - `fe-read*`: reads various formats of disk. See the per-format documentation + linked from the table above. These all take an optional `--write-flux` + option which will cause the raw flux to be written to the specified file. + + - `fe-write*`: writes various formats of disk. Again, see the per-format + documentation above. + + - `fe-writeflux`: writes raw flux files. This is much less useful than you + might think: you can't write flux files read from a disk to another disk. + (See the [FAQ](faq.md) for more information.) It's mainly useful for flux + files synthesised by the other `fe-write*` commands. + + - `fe-writetestpattern`: writes regular pulses (at a configurable interval) + to the disk. Useful for testing drive jitter, erasing disks in a more + secure fashion, or simply debugging. Goes well with `fe-inspect`. + + - `fe-rpm`: measures the RPM of the drive (requires a disk in the drive). + Mainly useful for testing. + + - `fe-seek`: moves the head. Mainly useful for finding out whether your drive + can seek to track 82. (Mine can't.) + + - `fe-testbulktransport`: measures your USB throughput. You need about 600kB/s + for FluxEngine to work. You don't need a disk in the drive for this one. + + - `fe-upgradefluxfile`: occasionally I need to upgrade the flux file format in + a non-backwards-compatible way; this tool will upgrade flux files to the new + format. + +Commands which normally take `--source` or `--dest` get a sensible default if left +unspecified. `fe-readibm` on its own will read drive 0 and write an `ibm.img` file. + +## Extra programs + +Supplied with FluxEngine, but not part of FluxEngine, are some little tools I +wrote to do useful things. These are built alongside FluxEngine. + + - `brother120tool`: extracts files from a 120kB Brother filesystem image. + + - `cwftoflux`: converts (one flavour of) CatWeasel flux file into a + FluxEngine flux file. + +## The recommended workflow + +So you've just received, say, a huge pile of old Brother word processor disks containing valuable historical data, and you want to read them. + +Typically I do this: + +``` +$ fe-readbrother -s :d=0 -o brother.img --write-flux=brother.flux +``` + +This will read the disk in drive 0 and write out a filesystem image. It'll also copy the flux to brother.flux. If I then need to tweak the settings, I can rerun the decode without having to physically touch the disk like this: + +``` +$ fe-readbrother -s brother.flux -o brother.img +``` + +If the disk is particularly fragile, you can force FluxEngine not to retry +failed reads with `--retries=0`. This reduces head movement. **This is not +recommended.** Floppy disks are inherently unreliable, and the occasional bit +error is perfectly normal; the sector will read fine next time. If you +prevent retries, then not only do you get bad sectors in the resulting image, +but the flux file itself contains the bad read, so attempting a decode of it +will just reproduce the same bad data.