salfter/fluxengine
1
0
Fork 0
mirror of https://github.com/davidgiven/fluxengine.git synced 2025-10-24 11:11:02 -07:00
Code Issues Packages Projects Releases Wiki Activity

Big documentation rework ouch my fingers.

Browse Source
This commit is contained in:
David Given
2019-03-07 22:55:43 +01:00
parent 82dbcb781c
commit eec6698996
8 changed files with 436 additions and 331 deletions
Download Patch File Download Diff File Expand all files Collapse all files

19
COPYING
View File

@@ -1,19 +0,0 @@
Copyright © 2007-2009 David Given
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in
all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
THE SOFTWARE.

19
COPYING.md Normal file
View File

@@ -0,0 +1,19 @@
Copyright © 2018-2019 David Given
- Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to
deal in the Software without restriction, including without limitation
the rights to use, copy, modify, merge, publish, distribute, sublicense,
and/or sell copies of the Software, and to permit persons to whom the
Software is furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included
in all copies or substantial portions of the Software.
- THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS
OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN
NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM,
DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR
OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE
USE OR OTHER DEALINGS IN THE SOFTWARE.

21
LICENSE
View File

@@ -1,21 +0,0 @@
MIT License
Copyright (c) 2018 David Given
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

336
README.md
View File

@@ -1,6 +1,9 @@
FluxEngine
==========
(If you're reading this on GitHub, the formatting's a bit messed up. [Try the
version on cowlark.com instead.](http://cowlark.com/fluxengine/)
What?
-----
@@ -10,78 +13,56 @@ PC drive to accept Amiga disks, CLV Macintosh disks, bizarre 128-sector CP/M
disks, and other weird and bizarre formats. (Although not all of these are
supported yet. I could really use samples.)
<img src="doc/floppy.jpg" style="width:50%" alt="a FluxEngine attached to a floppy drive">
The hardware consists of a single, commodity part with a floppy drive
connector soldered onto it. No ordering custom boards, no fiddly surface
mount assembly, and no fuss: nineteen simpler solder joints and you're done.
Don't believe me? Watch the demo reel!
<div style="text-align: center">
<iframe width="373" height="210" src="https://www.youtube.com/embed/m_s1iw8eW7o" frameborder="0" allow="accelerometer; autoplay; encrypted-media; gyroscope; picture-in-picture" allowfullscreen></iframe>
</div>
**Important note.** On 2019-02-09 I did a hardware redesign and moved the pins on
the board. Sorry for the inconvenience, but it means you don't have to modify
the board any more to make it work. If you built the hardware prior to then,
you'll need to adjust it.
**New!** There's a demo reel!
Where?
------
<div style="text-align: center">
<iframe width="560" height="315" src="https://www.youtube.com/embed/m_s1iw8eW7o" frameborder="0" allow="accelerometer; autoplay; encrypted-media; gyroscope; picture-in-picture" allowfullscreen></iframe>
</div>
It's [open source on GitHub!](https://github.com/davidgiven/fluxengine)
### Infrequently asked questions because nobody's got round to asking them yet
How?
----
**Q.** Why not just use a USB floppy drive? There are lots and they're cheap.
This page was getting kinda unwieldy so I've broken it up. Please consult the
following friendly articles:
**A.** Because USB floppy drives typically support a very limited set of
formats --- typically only IBM 1440kB and 720kB. The FluxEngine should work
on (almost) anything.
- [Frequently asked questions](doc/faq.md) ∾ but why...? ∾ does it...? ∾ can it...?
- [How the FluxEngine works](doc/technical.md) ∾ nitty gritty of the
sampler/sequencer hardware ∾ useful links on floppy drives ∾ why I'm not
using an Arduino/STM32/ESP32/Raspberry Pi
**Q.** But aren't floppy disks obsolete?
- [Making a FluxEngine](doc/building.md) ∾ what parts you need ∾ building it ∾
setting up the toolchain ∾ compiling the firmware ∾ programming the board
**A.** Absolutely they are. That doesn't mean they've gone away. Good luck
with any old hardware, for example; a classic Mac won't boot without a
classic Mac boot disk, and you can't make them on PCs (because they're
weird). This is where the FluxEngine comes in.
- [Using a FluxEngine](doc/using.md) ∾ what to do with your new hardware ∾
flux files and image files ∾ knowing what you're doing
**Q.** But how can you read and write non-PC formats on a PC floppy drive?
- [Reading dubious disks](doc/problems.md) ∾ it's not an exact science ∾
the sector map ∾ clock detection and the histogram ∾ tuning the clock ∾
manual adjustment
**A.** Because the FluxEngine hardware simply streams the raw magnetic flux
pulsetrain from the drive to the PC, and then the analysis is done off-line
in software. It doesn't rely on any floppy disk controller to interpret the
pulsetrain, so we can be a lot cleverer. In fact, the disk doesn't even have
to be spinning at the same speed.
Supported disk formats
----------------------
**Q.** Does it work on 5.25" drives?
The current support state is as follows.
**A.** Yes! Although PC 5.25" drives spin at 360 RPM rather than 300 RPM,
which means there's only 166ms of data on one per track rather than 200ms;
if you try to write a 3.5" format disk onto one it probably won't work.
**Q.** Does it work on 8" drives?
**A.** Probably? You'd need an adapter to let you connect the drive to the
FluxEngine --- [you can get them](http://www.dbit.com/fdadap.html). I don't
have either the adapter, the drive, or any 8" disks. If anyone wants to give
it a try, please [tell me about
it](https://github.com/davidgiven/fluxengine/issues/new).
**Q.** Is this like KryoFlux? Do you support KryoFlux stream files?
**A.** It's very like KryoFlux, although much simpler. Yes, FluxEngine can
read from KryoFlux stream files (but not write to them yet; nobody's asked).
FluxEngine doesn't capture all the data that KryoFlux does, like index
markers.
**Q.** I've tried it and my disk doesn't work!
**A.** [I have an entire page on diagnosing read failures.](doc/problems.md)
**Q.** That's awesome! What formats does it support?
**A.** I'm glad you asked the question. Consult the following table!
### Formats that it supports
Here's the table.
| Format | Read? | Write? | Notes |
|:---------------------------------------|:-----:|:------:|-------|
| IBM PC compatible | 🦄 | | and compatibles (like the Atari ST) |
| Format | Read? | Write? | Notes |
|:-----------------------------------------|:-----:|:------:|-------|
| IBM PC compatible | 🦄 | | and compatibles (like the Atari ST) |
| [Acorn ADFS](doc/disk-acornadfs.md) | 🦄 | | single- and double- sided |
| [Acorn DFS](doc/disk-acorndfs.md) | 🦄 | | |
| [AES Superplus / No Problem](doc/disk-aeslanier.md) | 🦖 | | hard sectors! and _very_ experimental |
@@ -132,240 +113,6 @@ most likely be able to read the data fine, unless they're doing bizarre
things like spiral tracks or partially encoded data, but let's stick with
normal conventionally formatted disks for the time being!
### Big list of stuff to work on
Both the firmware and the software are works in progress. There's still things
to do.
- Support for more floppy disk formats! I'm trying to scare up some sample
disks to image and decode; I'm particularly looking for Commodore 64 1541
disks and Apple Macintosh 800kB CLV disks. If you have any which I can
borrow and you live in (or near) Switzerland, please [get in
touch](https://github.com/davidgiven/fluxengine/issues/new).
- Better (and more) write support. The hardware can write disks, but not much
of the software's done. Writing is hard because I really need the device
to test the disks on. I could use some help here. Also, most of the write
code is a bit of a disaster and needs a major refactoring.
- Sourcing a different microcontroller. The PSoC5 is a great thing to work
with, but it's disappointingly proprietary and the toolchain only works on
Windows. It'd be nice to support something easier to work with. I need a 5V
microcontroller (which unfortunately rules out Arduinos) with at least
seventeen GPIO pins in a row. As non-PSoC5 microcontrollers won't have the
FPGA soft logic, that also means I'd most likely need to bitbang the floppy
drive, so speed is important. If you have any recommendations, please [get
in touch](https://github.com/davidgiven/fluxengine/issues/new).
Where?
------
It's [open source on GitHub!](https://github.com/davidgiven/fluxengine)
How?
----
### Introduction
The system is based around an off-the-shelf Cypress development board,
costing about $10 plus shipping. The only hardware modification you need to
do is to attach a floppy drive connector.
[I wrote a long, detailed set of instructions on building and programming
it.](doc/building.md)
I'm going to assume you've done this.
### Using it
1. Attach the FluxEngine to your floppy disk cable.
If you built your board using a connector, this is easy: just plug it in. If you're using pins, you need to make sure that the red strip on the cable is **to the right** (next to pin 2.7 on the board), and that the pins plug into the **lower set of holes** in the connector (so the connector overhangs the top of the board).
2. Attach the cable to your drives.
Floppy disk cables typically have [two pairs of floppy disk drive
connectors with a twist between
them](http://www.nullmodem.com/Floppy.htm). (Each pair has one connector
for a 3.5" drive and a different one for a 5.25" drive.) Normally the
first floppy disk drive is drive B, and the one after the twist is drive
A. However, FluxEngine considers the first drive to be drive 0 and the
second one to be drive 1. It's this way so as to allow the FluxEngine to
be plugged directly onto the back of a disk drive. If you only have one
drive, remember to plug the drive into the connector _before_ the twist.
(Or use `-s :d=1` to select drive 1 when working with disks.)
2. **Important.** Make sure that no disk you care about is in the drive.
(Because if your wiring is wrong and a disk is inserted, you'll probably
corrupt it.)
3. Connect the floppy drive to power. Nothing should happen. If anything
does, disconnect it and check step 1.
4. Connect the FluxEngine to your PC via USB --- using the little socket on
the board, not the big programmer plug.
5. Do `.obj/fe-rpm` from the shell. The motor should work and it'll tell you
that the disk is spinning at about 300 rpm. If it doesn't, please [get
in touch](https://github.com/davidgiven/fluxengine/issues/new).
6. Insert a standard PC formatted floppy disk into the drive (probably a good
idea to remove the old disk first). Then do `.obj/fe-readibm`. It should
read the disk, emitting copious diagnostics, and spit out an `ibm.img`
file containing the decoded disk image (either 1440kB or 720kB depending).
7. Success!
### The programs
#### Source and destination specifiers
When reading from or writing to _a disk_ (or a file pretending to be a disk),
use the `-s` and `-d` options to tell FluxEngine which bits of the disk you
want to access. These use a common syntax:
```
fe-readibm -s fakedisk.flux:t=0-79:s=0
```
- To access a real disk, leave out the filename (so `:t=0-79:s=0`).
- To access only some tracks, use the `t=` modifier. To access only some
sides, use the `s=` modifier. To change drives, use `d=`.
- Inside a modifier, you can use a comma separated list of ranges. So
`:t=0-3` and `:t=0,1,2,3` are equivalent.
- When specifying a range, you can also specify the step. For example,
`:t=0-79x2` would be used when accessing a 40-track disk with double
stepping.
- To read from drive 1 instead of drive 0, use `:d=1`.
- To read from a set of KryoFlux stream files, specify the path to the
directory containing the files _with a trailing slash_; so
`some/files/:t=0-10`. There must be a files for a single disk only
in the directory.
Source and destination specifiers work entirely in *physical units*.
FluxEngine is intended to be connected to an 80 (or 82) track double sided
drive, and these are the units used. If the format you're trying to access
lays out its tracks differently, then you'll need a specifier which tells
FluxEngine how to find those tracks. See the 40-track disk example above.
If you _don't_ specify a modifier, you'll get the default, which should be
sensible for the command you're using.
**Important note:** FluxEngine _always_ uses zero-based units (even if the
*disk format says otherwise).
### The commands
The FluxEngine client software is a largely undocumented set of small tools.
You'll have to play with them. They all support `--help`. They're not
installed anywhere and after building you'll find them in the `.obj`
directory.
- `fe-erase`: wipes (all or part of) a disk --- erases it without writing
a pulsetrain.
- `fe-inspect`: dumps the raw pulsetrain / bitstream to stdout. Mainly useful
for debugging.
- `fe-read*`: reads various formats of disk. See the per-format documentation
linked from the table above.
- `fe-write*`: writes various formats of disk. Again, see the per-format
documentation above.
- `fe-writeflux`: writes raw flux files. This is much less useful than you
might think: you can't necessarily copy flux files read from a disk,
because errors in the sampling are compounded and the result probably
isn't readable. It's mainly useful for flux files synthesised by the
other `fe-write*` commands.
- `fe-writetestpattern`: writes regular pulses (at a configurable interval)
to the disk. Useful for testing drive jitter, erasing disks in a more
secure fashion, or simply debugging. Goes well with `fe-inspect`.
- `fe-readibm`: reads 720kB or 1440kB IBM MFM disks. Emits a standard
filesystem image.
- `fe-rpm`: measures the RPM of the drive (requires a disk in the drive).
Mainly useful for testing.
- `fe-seek`: moves the head. Mainly useful for finding out whether your drive
can seek to track 82. (Mine can't.)
- `fe-testbulktransport`: measures your USB throughput. You need about 600kB/s
for FluxEngine to work. You don't need a disk in the drive for this one.
- `fe-upgradefluxfile`: occasionally I need to upgrade the flux file format in
a non-backwards-compatible way; this tool will upgrade flux files to the new
format.
Commands which take `--source` or `--dest` take a parameter of the syntax
`$FILENAME:$MODIFIER:$MODIFER...` as described above. If left unspecified,
you get the default specified by the command, which will vary depending on
which disk format you're using (and is usually the right one).
### How it works
It's very very simple. The firmware measures the time between flux transition
pulses, encodes them, and streams them over USB to the PC.
There's an 8-bit counter attached to an 12MHz clock. This is used to measure
the interval between pulses. If the timer overflows, we pretend it's a pulse
(this very rarely happens in real life). ([I'm working on
this.](https://github.com/davidgiven/fluxengine/issues/8))
An HD floppy has a nominal clock of 500kHz, so we use a sample clock of 12MHz
(every 83ns). This means that our 500kHz pulses will have an interval of 24
(and a DD disk with a 250kHz nominal clock has an interval of 48). This gives
us more than enough resolution. If no pulse comes in, then we sample on
rollover at 21us.
(The clock needs to be absolutely rock solid or we get jitter which makes the
data difficult to analyse, so 12 was chosen to be derivable from the
ultra-accurate USB clock.)
Once at the PC, we do some dubious heuristics to determine the clock rate,
which depends on what kind of encoding is being used, and that in turn lets
us decode the pulsetrain into actual bits and derive the raw floppy disk
records from there. Reassembling the image and doing stuff like CRC checking
is straightforward.
Some useful and/or interesting numbers:
- nominal rotation speed is 300 rpm, or 5Hz. The period is 200ms.
- a pulse is 150ns to 800ns long.
- a 12MHz tick is 83ns.
- MFM HD encoding uses a clock of 500kHz. This makes each recording cell 2us,
or 24 ticks. For DD it's 4us and 48 ticks.
- a short transition is one cell (2us == 24 ticks). A medium is a cell and
a half (3us == 36 ticks). A long is two cells (4us == 48 ticks). Double
that for DD.
- pulses are detected with +/- 350ns error for HD and 700ns for DD. That's
4 ticks and 8 ticks. That seems to be about what we're seeing.
- in real life, start going astray after about 128 ticks == 10us. If you
don't write anything, you read back random noise.
Useful links:
- [The floppy disk user's
guide](http://www.hermannseib.com/documents/floppy.pdf): an incredibly
useful compendium of somewhat old floppy disk information --- which is
fine, because floppy disks are somewhat old.
- [The TEAC FD-05HF-8830 data
sheet](https://hxc2001.com/download/datasheet/floppy/thirdparty/Teac/TEAC%20FD-05HF-8830.pdf):
the technical data sheet for a representative drive. Lots of useful
timing numbers here.
- [KryoFlux stream file
documentation](https://www.kryoflux.com/download/kryoflux_stream_protocol_rev1.1.pdf):
the format of KryoFlux stream files (partially supported by FluxEngine)
Who?
----
@@ -376,6 +123,13 @@ There may or may not be anything interesting there.
License
-------
Everything here is licensed under the MIT open source license. Please see
Everything here _except the contents of the `dep` directory_ is © 2019 David
Given and is licensed under the MIT open source license. Please see
[COPYING](COPYING) for the full text. The tl;dr is: you can do what you like
with it provided you don't claim you wrote it.
As an exception, `dep/fmt` contains a copy of [fmt](http://fmtlib.net),
maintained by Victor Zverovich (`vitaut <https://github.com/vitaut>`_) and
Jonathan Müller (`foonathan <https://github.com/foonathan>`_) with
contributions from many other people. It is licensed under the terms of the
BSD license. Please see the contents of the directory for the full text.

59
doc/faq.md Normal file
View File

@@ -0,0 +1,59 @@
Frequently asked questions
==========================
**Q.** Why not just use a USB floppy drive? There are lots and they're cheap.
**A.** Because USB floppy drives typically support a very limited set of
formats --- typically only IBM 1440kB and 720kB. The FluxEngine should work
on (almost) anything, including the ones that IBM machines won't touch. Also,
as it's USB, it'll work happily on machines that were never designed for
floppy disks (like my development Chromebook).
**Q.** But aren't floppy disks obsolete?
**A.** Absolutely they are. That doesn't mean they've gone away. Good luck
with any old hardware, for example; a classic Mac won't boot without a
classic Mac boot disk, and you can't make them on PCs (because they're
weird). This is where the FluxEngine comes in.
**Q.** But how can you read and write non-PC formats on a PC floppy drive?
**A.** Because the FluxEngine hardware simply streams the raw magnetic flux
pulsetrain from the drive to the PC, and then the analysis is done off-line
in software. It doesn't rely on any floppy disk controller to interpret the
pulsetrain, so we can be a lot cleverer. In fact, the disk doesn't even have
to be spinning at the same speed.
**Q.** Does it work on 5.25" drives?
**A.** Yes! Although PC 5.25" drives spin at 360 RPM rather than 300 RPM,
which means there's only 166ms of data on one per track rather than 200ms;
if you try to write a 3.5" format disk onto one it probably won't work.
**Q.** Does it work on 8" drives?
**A.** Probably? You'd need an adapter to let you connect the drive to the
FluxEngine --- [you can get them](http://www.dbit.com/fdadap.html). I don't
have either the adapter, the drive, or any 8" disks. If anyone wants to give
it a try, please [tell me about
it](https://github.com/davidgiven/fluxengine/issues/new).
**Q.** Is this like KryoFlux / Catweasel / DiskFerret? Do you support KryoFlux stream files?
**A.** It's very like all of these; the idea's old, and lots of people have
*tried it (you can get away with any sufficiently fast microcontroller and
*enough RAM). FluxEngine can
read from KryoFlux stream files natively, and there's a tool which will let
you convert at least one kind of Catweasel file to FluxEngine's native flux
file format.
**Q.** Can I use this to make exact copies of disks?
**A.** No. FluxEngine can read disks, and it can write disks, but it can't
write flux files that it read. There's several reasons for this, including
but not limited to: amplification of disk noise causing unreadable disks;
needing to rearrange the read data so it all fits exactly between the index
markers; reproduction of non-repeatable noise (this was a common trick with
Apple II copy protection); etc. What FluxEngine prefers is to read a disk,
turn it into a filesystem image, and then synthesise flux from the filesystem
image and write _that_ to another disk.

BIN
doc/floppy.jpg
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 120 KiB

123
doc/technical.md Normal file
View File

@@ -0,0 +1,123 @@
How the FluxEngine works
========================
## The hardware design
It's essentially very simple, although the details are complicated. In
essence, all the firmware does is to measure the time between flux transition
pulses and send them to the PC.
There's an 8-bit counter attached to an 12MHz clock. This is used to measure
the interval between pulses. A Cypress datapath state machine thingy using
Verilog turns the timer, pulse and index information into a bytecode stream
which encodes intervals, pulses, and whether the index hole has been seen.
This is then streamed back to the PC, where offline software decodes it: it
does simple statistical analysis to guess the clock rate, then turns the
pulses into a nice, lined up bit array and from there decodes the file system
format.
Writing back to disk works the same way: bytes are streamed to the
FluxEngine, where a different datapath state machine thingy (the PSoC5LP has
24, all independently programmable) to interpret the bytecodes, generate a
stream of pulses to the disk.
The bytecode format represents an interval between pulses as a byte, a pulse
as a byte, and the index hole as a byte. Timer overflows are handled by
sending multiple intervals in a row.
An HD floppy has a nominal pulse frequency of 500kHz, and we use a sample
clock of 12MHz (every 83ns). This means that our 500kHz pulses will have an
average interval of 24. This gives us more than enough resolution. At this
speed, in the 200ms that a 3.5" disk takes to rotate, we will see about
100,000 pulses. Each one is encoded as two bytes, one for the interval and
one to generate the pulse; so that revolution generates 200kB of data.
(Extremely approximately. The actual figure is less.)
(The clock needs to be absolutely rock solid or we get jitter which makes the
data difficult to analyse, so 12 was chosen to be derivable from the
ultra-accurate USB clock.)
Some useful and/or interesting numbers:
- nominal rotation speed is 300 rpm, or 5Hz. The period is 200ms.
- a pulse, as generated by the floppy drive electronics, is 150ns to 800ns
long.
- a 12MHz tick is 83ns.
- MFM HD encoding uses a clock of 500kHz. This makes each recording cell 2us,
or 24 ticks. For DD it's 4us and 48 ticks.
- a short transition is one cell (2us == 24 ticks). A medium is a cell and
a half (3us == 36 ticks). A long is two cells (4us == 48 ticks). Double
that for DD.
- pulses are detected with +/- 350ns error for HD and 700ns for DD. That's
4 ticks and 8 ticks. That seems to be about what we're seeing.
- in real life, pulses start going astray on a 3.5" drive after about 128
ticks == 10us. I haven't tried with a 5.25" drive yet as I don't have a
5.25" scratch disk.
## Why don't I use an Arduino / STM32 / ESP32 / Raspberry Pi / etc?
I've got a _lot_ of questions on this, and multiple Github issues of people
debating it. It's complicated, but it's essentially a tradeoff between speed
and complexity.
FluxEngine's read process involves generating a lot of data using a fairly
brute force sampling approach --- about 150kB per disk revolution, and
sometimes it needs to record multiple revolutions. Most microcontrollers
don't have enough RAM to buffer this, so instead I have to stream it over USB
back to the host PC in real time. The disk won't wait, so I need to stream data faster
than the disk is producing it: the total is about 800kB/s.
Handling USB is pretty CPU-hungry, so my candidate microntroller has to be
able to cope with the ruinously strict real-time requirements of the
sampler's 12MHz clock as well as keeping up with 13,000 USB interrupts a
second (one for each 64-byte frame) in order to transfer the data.
The Atmels and STM32s I found were perfectly capable of doing the real-time
sampling, using hand-tool assembly, but I very much doubt whether they could
do the USB streaming as well (although I'd like to move away from the Cypress
onto something less proprietary and easier to source, so I'd like to be
proven wrong here).
The Raspberry Pi easily has enough processing power and memory, but it's also
got terrible GPIO pin read performance --- [about
1kHz](https://raspberrypi.stackexchange.com/questions/9646/how-fast-is-gpiodma-multi-i2s-input/10197#10197).
That's a long way from the 12MHz I need.
The PSoC5LP part I'm using has enough CPU to handle the USB side of things,
and it _also_ has a whole set of FPGA-like soft programmable features,
including 24 mini-ALU systems that are ideally suited to exactly this kind of
sampling. I can read the disk and generate the byte stream describing the
flux pattern entirely in 'hardware', without involving the main CPU at all.
This is then DMAed directly into a set of ring buffers read for the USB
system to pick up and relay back to the PC. It's incredibly simple and works
well. (The same applies to writing flux back onto the disk.)
The development board I'm using, the
[CY8CKIT-059](https://www.cypress.com/documentation/development-kitsboards/cy8ckit-059-psoc-5lp-prototyping-kit-onboard-programmer-and),
also has another big advantage: it's the right shape. It's got 17 holes in a
row connected to GPIO pins which means I can just connect a floppy drive
connector directly to the board without needing to build any hardware. No
adapter board, no special cable, nothing. This makes the FluxEngine hardware
incredibly easy to assemble, which therefore means cheap.
Speaking of which, the CY8CKIT-059 is $10. (Before shipping, which is
admittedly expensive.)
### Some useful links
- [The floppy disk user's
guide](http://www.hermannseib.com/documents/floppy.pdf): an incredibly
useful compendium of somewhat old floppy disk information --- which is
fine, because floppy disks are somewhat old.
- [The TEAC FD-05HF-8830 data
sheet](https://hxc2001.com/download/datasheet/floppy/thirdparty/Teac/TEAC%20FD-05HF-8830.pdf):
the technical data sheet for a representative drive. Lots of useful
timing numbers here.
- [KryoFlux stream file
documentation](https://www.kryoflux.com/download/kryoflux_stream_protocol_rev1.1.pdf):
the format of KryoFlux stream files (partially supported by FluxEngine)

190
doc/using.md Normal file
View File

@@ -0,0 +1,190 @@
Using a FluxEngine
==================
So you've [built the hardware](building.md)! What now?
## Connecting it up
In order to do anything useful, you have to plug it in to a floppy disk drive (or two).
1. Plug the motherboard end of your floppy disk cable into the FluxEngine.
The **red stripe goes on the right**. The **lower set of
holes connect to the board**. (Pin 2 of the connector needs to connect
to pin 2.7 on the board.)
If you're using header pins, the upper row of holes in the connector
should overhang the edge of the board. If you're using a floppy drive
motherboard connector, you're golden, of course (unless you have one of
those annoying unkeyed cables, or have accidentally soldered the
connector on in the wrong place --- don't laugh, I've done it.)
2. Plug the drive end of your floppy disk cable into the drive (or drives).
Floppy disk cables typically have [two pairs of floppy disk drive
connectors with a twist between
them](http://www.nullmodem.com/Floppy.htm). (Each pair has one connector
for a 3.5" drive and a different one for a 5.25" drive.) (Some cables
are cheap and just have the 3.5" connectors. Some are _very_ cheap and
have a single 3.5" connector, after the twist.)
FluxEngine uses, sadly, non-standard disk numbering (there are reasons).
Drive 0 is the one nearest the motherboard; that is, before the twist.
Drive 1 is the one at the end of the cable; that is, after the twist.
Drive 0 is the default. If you only have one drive, remember to plug the
drive into the connector _before_ the twist. (Or use `-s :d=1` to select
drive 1 when working with disks.)
3. **Important.** Make sure that no disk you care about is in the drive.
(Because if your wiring is wrong and a disk is inserted, you'll corrupt it.)
4. Connect the floppy drive to power. Nothing should happen. If you've
connected something in backwards, you'll see the drive light up, the
motor start, and if you didn't take the disk out, one track has just
been wiped. If this happens, check your wiring.
5. Connect the FluxEngine to your PC via USB --- using the little socket on
the board, not the big programmer plug.
6. Insert a scratch disk and do `.obj/fe-rpm` from the shell. The motor
should work and it'll tell you that the disk is spinning at about 300
rpm for a 3.5" disk, or 360 rpm for a 5.25" disk. If it doesn't, please
[get in touch](https://github.com/davidgiven/fluxengine/issues/new).
7. Do `.obj/fe-testbulktransport` from the shell. It'll measure your USB
bandwidth. Ideally you should be getting above 900kB/s. FluxEngine needs
about 850kB/s, so if you're getting less than this, try a different USB
port.
8. Insert a standard PC formatted floppy disk into the drive (probably a good
idea to remove the old disk first). Then do `.obj/fe-readibm`. It should
read the disk, emitting copious diagnostics, and spit out an `ibm.img`
file containing the decoded disk image (either 1440kB or 720kB depending).
9. Profit!
## The programs
I'm sorry to say that the programs are very badly documented --- they're
moving too quickly for the documentation to keep up. They do all respond to
`--help`. There are some common properties, described below.
### Source and destination specifiers
When reading from or writing to _a disk_ (or a file pretending to be a disk),
use the `--source` (`-s`) and `--dest` (`-d`) options to tell FluxEngine
which bits of the disk you want to access. These use a common syntax:
```
.obj/fe-readibm -s fakedisk.flux:t=0-79:s=0
```
- To access a real disk, leave out the filename (so `:t=0-79:s=0`).
- To access only some tracks, use the `t=` modifier. To access only some
sides, use the `s=` modifier. To change drives, use `d=`.
- Inside a modifier, you can use a comma separated list of ranges. So
`:t=0-3` and `:t=0,1,2,3` are equivalent.
- When specifying a range, you can also specify the step. For example,
`:t=0-79x2` would be used when accessing a 40-track disk with double
stepping.
- To read from drive 1 instead of drive 0, use `:d=1`.
- To read from a set of KryoFlux stream files, specify the path to the
directory containing the files _with a trailing slash_; so
`some/files/:t=0-10`. There must be a files for a single disk only
in the directory.
Source and destination specifiers work entirely in *physical units*.
FluxEngine is intended to be connected to an 80 (or 82) track double sided
drive, and these are the units used. If the format you're trying to access
lays out its tracks differently, then you'll need a specifier which tells
FluxEngine how to find those tracks. See the 40-track disk example above.
If you _don't_ specify a modifier, you'll get the default, which should be
sensible for the command you're using.
**Important note:** FluxEngine _always_ uses zero-based units (even if the
*disk format says otherwise).
### The commands
The FluxEngine client software is a largely undocumented set of small tools.
You'll have to play with them. They all support `--help`. They're not
installed anywhere and after building you'll find them in the `.obj`
directory.
- `fe-erase`: wipes (all or part of) a disk --- erases it without writing
a pulsetrain.
- `fe-inspect`: dumps the raw pulsetrain / bitstream to stdout. Mainly useful
for debugging.
- `fe-read*`: reads various formats of disk. See the per-format documentation
linked from the table above. These all take an optional `--write-flux`
option which will cause the raw flux to be written to the specified file.
- `fe-write*`: writes various formats of disk. Again, see the per-format
documentation above.
- `fe-writeflux`: writes raw flux files. This is much less useful than you
might think: you can't write flux files read from a disk to another disk.
(See the [FAQ](faq.md) for more information.) It's mainly useful for flux
files synthesised by the other `fe-write*` commands.
- `fe-writetestpattern`: writes regular pulses (at a configurable interval)
to the disk. Useful for testing drive jitter, erasing disks in a more
secure fashion, or simply debugging. Goes well with `fe-inspect`.
- `fe-rpm`: measures the RPM of the drive (requires a disk in the drive).
Mainly useful for testing.
- `fe-seek`: moves the head. Mainly useful for finding out whether your drive
can seek to track 82. (Mine can't.)
- `fe-testbulktransport`: measures your USB throughput. You need about 600kB/s
for FluxEngine to work. You don't need a disk in the drive for this one.
- `fe-upgradefluxfile`: occasionally I need to upgrade the flux file format in
a non-backwards-compatible way; this tool will upgrade flux files to the new
format.
Commands which normally take `--source` or `--dest` get a sensible default if left
unspecified. `fe-readibm` on its own will read drive 0 and write an `ibm.img` file.
## Extra programs
Supplied with FluxEngine, but not part of FluxEngine, are some little tools I
wrote to do useful things. These are built alongside FluxEngine.
- `brother120tool`: extracts files from a 120kB Brother filesystem image.
- `cwftoflux`: converts (one flavour of) CatWeasel flux file into a
FluxEngine flux file.
## The recommended workflow
So you've just received, say, a huge pile of old Brother word processor disks containing valuable historical data, and you want to read them.
Typically I do this:
```
$ fe-readbrother -s :d=0 -o brother.img --write-flux=brother.flux
```
This will read the disk in drive 0 and write out a filesystem image. It'll also copy the flux to brother.flux. If I then need to tweak the settings, I can rerun the decode without having to physically touch the disk like this:
```
$ fe-readbrother -s brother.flux -o brother.img
```
If the disk is particularly fragile, you can force FluxEngine not to retry
failed reads with `--retries=0`. This reduces head movement. **This is not
recommended.** Floppy disks are inherently unreliable, and the occasional bit
error is perfectly normal; the sector will read fine next time. If you
prevent retries, then not only do you get bad sectors in the resulting image,
but the flux file itself contains the bad read, so attempting a decode of it
will just reproduce the same bad data.