Typo fixes.

Add the drive response documentation page.

README.md

@@ -64,9 +64,11 @@ following friendly articles:
 	software](doc/greaseweazle.md) ∾ what works ∾ what doesn't work ∾ where to
 	go for help
 
+  - [Troubleshooting dubious disks](doc/problems.md) ∾ it's not an exact
+	science ∾ the sector map ∾ clock detection and the histogram
 
-  - [Troubleshooting dubious disks](doc/problems.md) ∾ it's not an exact science ∾
-    the sector map ∾ clock detection and the histogram
+  - [Checking your drive](doc/driveresponse.md) ∾ you can't do that with that ∾
+	measuring your drive's ability to work with exotic formats
 
 Which?
 ------

doc/driveresponse.md

@@ -0,0 +1,81 @@
+Analysing drive response
+========================
+
+Not all PC drives are made equal. Some are less equal than others.
+
+The way floppy disk storage works is that the floppy drive controller will
+generate a series of pulses, which the drive stores on the disk. Then, when the
+disk is read, the drive will reproduce the same series of pulses and return it
+to the floppy drive controller. The data is stored in the intervals between
+pulses.
+
+The problem is that some PC drives assume that they're going to be used with
+IBM scheme disks, which use particular pulse intervals --- in the case of DD
+disks, intervals are always 4us, 6us or 8us. So, in a misguided attempt to
+improve reliability, they sometimes... tidy... the incoming pulse stream. This
+can have nasty effects if you're not a disk which _doesn't_ use those intervals.
+
+In addition, they won't work properly if the intervals are too great, or too
+small. Partly this is a limitation of the underlying physics of the magnetic
+media, and partly it's due to the drive's automatic gain adjustment: if the
+drive doesn't see a pulse, it'll start ramping up the gain of its amplifier,
+until it starts interpreting random noise as a valid pulse.
+
+FluxEngine has a tool to analyse a drive and report on this behaviour. It works
+by writing a sequence of timed pulses to the disk, then reading them back and
+seeing what the drive actually reports. To use it, do:
+
+```
+fluxengine analyse driveresponse -d :d=1:t=0 --min-interval-us=0 --max-interval-us=30 --interval-step-us=.1 --write-csv=driveresponse.csv
+python3 scripts/driveresponse.csv
+```
+
+This will scan all intervals from 0us to 30us, at 0.1us steps, and write the
+result as a CSV file. Then the Python script uses matplotlib to render the
+result as a heatmap. They look like this.
+
+(Click to expand)
+
+<div style="text-align: center">
+<a href="sony-mpf920-dd.png"><img src="sony-mpf920-dd.png" style="width:40%" alt="Sony MPF-920, DD"></a>
+<a href="sony-mpf920-hd.png"><img src="sony-mpf920-hd.png" style="width:40%" alt="Sony MPF-920, HD"></a>
+</div>
+
+This is the analysis from the [Sony
+MPF-920](https://docs.sony.com/release/MPF920Z.pdf) 3.5" drive I mostly use for
+testing. The left-hand image shows the result from a DD disk, while the right
+hand image shows the result from a HD disk.
+
+The vertical access is the width of pulse being written; the horizontal axis
+and heatmap shows the distribution of pulses being read back. Yoou can see the
+diagonal line, which represents correct pulses. The triangular smear in the top
+left shows spurious pulses which are being read back because the interval is
+too great; these start at about 12us for DD disks and 7us for HD disks. This is
+an artifact of the different magnetic media for the two types of disk.
+
+(This, by the way, is why you shouldn't use DD formats on HD disks. The
+intervals on a DD disk can go up to 8us, which is on the edge of the ability of
+an HD disk and drive to correctly report back the pulses.)
+
+You also note the hard cut-off on the left: this represents the filter on the
+drive, which will simply refuse to report pulse intervals shorter than about
+1.5us. FluxEngine itself can't write intervals shorter than 2us.
+
+For comparison purposes, here's another set of graphs.
+
+<div style="text-align: center">
+<a href="fdd-90206-dd.png"><img src="fdd-90206-dd.png" style="width:40%" alt="FDD-90206, DD"></a>
+<a href="fdd-90206-hd.png"><img src="fdd-90206-hd.png" style="width:40%" alt="FDD-90206, HD"></a>
+</div>
+
+This is from another drive I have; it's an unbranded combo
+card-reader-and-floppy drive unit; the 90206 is the only identification mark it
+has. I don't use this because it's problematic, and the graph shows why; you
+can just see some ghosting on the HD graph at at 3us, where some pulses are
+coming back reported at 6us. This won't affect IBM scheme disks because they
+don't use 3us as an interval, but it might effect other formats. And the DD
+graph shows that intervals below about 4us are reported as double what they
+should be: so, this drive won't work on [Macintosh 800kB
+formats](disk-macintosh.md) at all, because they use intervals starting at
+2.6us, below this limit. But it should work on PC formats --- just.
+

doc/greaseweazle.md

@@ -24,7 +24,7 @@ List All Devices, and then open the big dropdown box and select the
 GreaseWeazle. You should see something like this.
 
 <div style="text-align: center">
-<img src="zadig.png" style="width:80%" alt="Zadig screenshot"></a>
+<img src="zadig.png" style="width:80%" alt="Zadig screenshot">
 </div>
 
 Ensure that the Driver boxes say `usbser` and `WinUSB`. Then press 'Replace

doc/problems.md

@@ -10,6 +10,11 @@ been written to be largely fire-and-forget and is mostly self-adjusting.
 However, there are still some things that can be tuned to produce better
 reads.
 
+Also, it's important to remember that some drives are problematic --- in
+particular, some 3.5" floppy drives are designed to work with just IBM scheme
+disks with a certain set of pulse intervals. There's a tool to let you diagnose
+this; see [the drive response](driveresponse.md) page.
+
 The sector map
 --------------
 

mkninja.sh

@@ -212,6 +212,7 @@ buildlibrary libbackend.a \
     lib/writer.cc \
 
 buildlibrary libfrontend.a \
+	src/fe-analysedriveresponse.cc \
     src/fe-cwftoflux.cc \
     src/fe-erase.cc \
     src/fe-fluxtoau.cc \

scripts/analysedriveresponse.py

@@ -0,0 +1,36 @@
+import numpy
+import matplotlib.pyplot as plt
+import matplotlib.animation as animation
+
+TICK_FREQUENCY = 12e6
+TICKS_PER_US = TICK_FREQUENCY / 1e6
+print(TICKS_PER_US)
+
+# Load data.
+data = numpy.loadtxt(open("driveresponse.csv", "rb"), delimiter=",", skiprows=1)
+
+labels = data[:, 0]
+frequencies = data[:, 1:]
+
+# Scale the frequencies.
+def scaled(row):
+    m = row.mean()
+    if m != 0:
+        return row / m
+    else:
+        return row
+
+scaledfreq = numpy.array([scaled(row) for row in frequencies])
+
+# Create new Figure with black background
+fig = plt.figure(figsize=(8, 8), facecolor='#aaa')
+
+plt.imshow(scaledfreq, extent=[0, 512/TICKS_PER_US, labels[0], labels[-1]], cmap='jet',
+           vmin=0, vmax=1, origin='lower', aspect='auto')
+plt.colorbar()
+plt.ylabel("Interval period (us)")
+plt.xlabel("Response (us)")
+plt.grid(True, dashes=(2, 2))
+plt.show()
+
+plt.show()

src/fe-analysedriveresponse.cc

@@ -0,0 +1,135 @@
+#include "globals.h"
+#include "flags.h"
+#include "usb/usb.h"
+#include "dataspec.h"
+#include "fluxmap.h"
+#include "decoders/fluxmapreader.h"
+#include "writer.h"
+#include "protocol.h"
+#include "fmt/format.h"
+#include <fstream>
+
+static FlagGroup flags = {
+	&usbFlags,
+};
+
+static DataSpecFlag dest(
+    { "--dest", "-d" },
+    "destination to analyse",
+    ":d=0:t=0:s=0");
+
+static DoubleFlag minInterval(
+	{ "--min-interval-us" },
+	"Minimum pulse interval",
+	2.0);
+
+static DoubleFlag maxInterval(
+	{ "--max-interval-us" },
+	"Maximum pulse interval",
+	10.0);
+
+static DoubleFlag intervalStep(
+	{ "--interval-step-us" },
+	"Interval step, approximately",
+	0.2);
+
+static StringFlag writeCsv(
+	{ "--write-csv" },
+	"Write detailed CSV data",
+	"driveresponse.csv");
+
+int mainAnalyseDriveResponse(int argc, const char* argv[])
+{
+    flags.parseFlags(argc, argv);
+
+    FluxSpec spec(dest);
+	if (spec.locations.size() != 1)
+		Error() << "the destination dataspec must contain exactly one track (two sides count as two tracks)";
+
+    usbSetDrive(spec.drive, false, F_INDEX_REAL);
+	usbSeek(spec.locations[0].track);
+
+	std::cout << "Measuring rotational speed...\n";
+    nanoseconds_t period = usbGetRotationalPeriod(0);
+	if (period == 0)
+		Error() << "Unable to measure rotational speed (try fluxengine rpm).";
+
+	std::ofstream csv;
+	if (writeCsv.get() != "")
+		csv.open(writeCsv);
+
+	for (double interval = minInterval; interval<maxInterval; interval += intervalStep)
+	{
+		unsigned ticks = (unsigned) (interval * TICKS_PER_US);
+		std::cout << fmt::format("Interval {:.2f}: ", ticks * US_PER_TICK);
+		std::cout << std::flush;
+
+		std::vector<int> frequencies(512);
+
+		if (interval >= 2.0)
+		{
+			/* Write the test pattern. */
+
+			Fluxmap outFluxmap;
+			while (outFluxmap.duration() < period)
+			{
+				outFluxmap.appendInterval(ticks);
+				outFluxmap.appendPulse();
+			}
+			usbWrite(spec.locations[0].side, outFluxmap.rawBytes(), 0);
+
+			/* Read the test pattern in again. */
+
+			Fluxmap inFluxmap;
+			inFluxmap.appendBytes(usbRead(spec.locations[0].side, true, period, 0));
+
+			/* Compute histogram. */
+
+			FluxmapReader fmr(inFluxmap);
+			fmr.seek((double)period*0.1); /* skip first 10% and last 10% as contains junk */
+			fmr.findEvent(F_BIT_PULSE);
+			while (fmr.tell().ns() < ((double)period*0.9))
+			{
+				unsigned ticks = fmr.findEvent(F_BIT_PULSE);
+				if (ticks < frequencies.size())
+					frequencies[ticks]++;
+			}
+		}
+
+		/* Compute standard deviation. */
+
+		int sum = 0;
+		int prod = 0;
+		for (int i=0; i<frequencies.size(); i++)
+		{
+			sum += frequencies[i];
+			prod += i * frequencies[i];
+		}
+		if (sum == 0)
+			std::cout << "failed\n";
+		else
+		{
+			double mean = prod / sum;
+			double sqsum = 0;
+			for (int i=0; i<frequencies.size(); i++)
+			{
+				double dx = (double)i - mean;
+				sqsum += (double)frequencies[i] * dx * dx;
+			}
+			double stdv = sqrt(sqsum / sum);
+			std::cout << fmt::format("{:.4f} {:.4f}\n", stdv, mean/TICKS_PER_US);
+
+		}
+
+		if (writeCsv.get() != "")
+		{
+			csv << interval;
+			for (int i : frequencies)
+				csv << "," << i;
+			csv << '\n';
+		}
+	}
+
+    return 0;
+}
+

src/fluxengine.cc

@@ -2,6 +2,7 @@
 
 typedef int command_cb(int agrc, const char* argv[]);
 
+extern command_cb mainAnalyseDriveResponse;
 extern command_cb mainErase;
 extern command_cb mainConvertCwfToFlux;
 extern command_cb mainConvertFluxToAu;
@@ -50,6 +51,7 @@ struct Command
 static command_cb mainRead;
 static command_cb mainWrite;
 static command_cb mainConvert;
+static command_cb mainAnalyse;
 static command_cb mainTest;
 
 static std::vector<Command> commands =
@@ -57,6 +59,7 @@ static std::vector<Command> commands =
     { "erase",             mainErase,             "Permanently but rapidly erases some or all of a disk." },
     { "convert",           mainConvert,           "Converts various types of data file.", },
     { "inspect",           mainInspect,           "Low-level analysis and inspection of a disk." },
+	{ "analyse",           mainAnalyse,           "Disk and drive analysis tools." },
     { "read",              mainRead,              "Reads a disk, producing a sector image.", },
     { "rpm",               mainRpm,               "Measures the disk rotational speed.", },
     { "seek",              mainSeek,              "Moves the disk head.", },
@@ -107,6 +110,11 @@ static std::vector<Command> convertables =
 	{ "image",         mainConvertImage,     "Converts one disk image to another.", },
 };
 
+static std::vector<Command> analysables =
+{
+	{ "driveresponse", mainAnalyseDriveResponse, "Measures the drive's ability to read and write pulses.", }
+};
+
 static std::vector<Command> testables =
 {
     { "bandwidth",     mainTestBandwidth, "Measures your USB bandwidth.", },
@@ -153,6 +161,9 @@ static int mainWrite(int argc, const char* argv[])
 static int mainConvert(int argc, const char* argv[])
 { return mainExtended(convertables, "convert", argc, argv); }
 
+static int mainAnalyse(int argc, const char* argv[])
+{ return mainExtended(analysables, "analyse", argc, argv); }
+
 static int mainTest(int argc, const char* argv[])
 { return mainExtended(testables, "test", argc, argv); }