Schneller Start ohne RubyGems, Rational statt BigDecimal, Helfer in Rdb::Helpers

bin/rdb läuft mit --disable-gems (ca. 85 statt 170 ms je Befehl), dafür
rechnen Summen und /act mit Rational. Hilfsfunktionen und Konstanten liegen
nicht mehr auf Object, Kopf-/Strichzeile und awk-Aufruf an einer Stelle.
Tests laufen parallel: 21 statt 70 Sekunden.

CHANGELOG.md

@@ -20,7 +20,11 @@ folgen [Semantic Versioning](https://semver.org/lang/de/). Die Version steht in
 ### Verbessert
 - Schnellzugriff: `search`, `seek`, `delete` und `update.rdb` lesen bei `-mh`, `-mi` und `-mr` nur noch den Indexeintrag und den einen Satz (Seek auf Byte-Positionen, bei `.i` Halbierung über die Datei), nicht mehr Tabelle und Index vollständig. `Data` liest die Zeilen erst, wenn eine Methode alle braucht.
 - Die Hashfunktion ist FNV-1a statt der Bytesumme. Vorher fielen 3000 fünfstellige Nummern in 30 Plätze (größter Eimer 223), jetzt liegt der größte Eimer bei etwa 4. **Hashindizes (`.h`) müssen neu gebaut werden** (`index -mh`).
-- `total`, `mean`, `subtotal` und die /act-Befehle rechnen mit `BigDecimal` statt `Float`: `0.1 + 0.2 + 0.7` ergibt `1.0`, Beträge summieren sich auf den Rappen genau, und es wird kaufmännisch gerundet (halb auf).
+- `total`, `mean`, `subtotal` und die /act-Befehle rechnen exakt mit `Rational` statt `Float`: `0.1 + 0.2 + 0.7` ergibt `1.0`, Beträge summieren sich auf den Rappen genau, und es wird kaufmännisch gerundet (halb auf).
+
+- `bin/rdb` startet mit `ruby --disable-gems`: jeder Befehl ist in etwa doppelt so schnell gestartet (ca. 85 statt 170 ms), weil RubyGems nicht geladen wird. `rdb` braucht nur die Standardbibliothek.
+- Die Tests laufen parallel (`parallelize_me!`; `MT_CPU=1` für nacheinander). Zusammen mit dem schnelleren Start dauert die Suite 21 statt 70 Sekunden.
+- Intern: Hilfsfunktionen der Befehlsdateien liegen in `Rdb::Helpers` statt auf `Object`, die Konstanten (`STATS`, `TERM_CAPS` ...) in `Rdb`. Kopf- und Strichzeile entstehen an einer Stelle (`Table.head_text`, `dash_cells`, `headed?`), `awk` baut auf `awk_text` auf.
 
 ### Sicherheit
 - `report`: Spaltenwerte in `<!befehl <Spalte>!>` wurden unmaskiert in einen Shell-Aufruf eingesetzt, eine Zelle konnte so Befehle ausführen. Sie werden jetzt mit `Shellwords.escape` eingesetzt.

README.md

@@ -246,7 +246,7 @@ er nicht mehr passt (außer mit `-a`). Beide halten die Indizes aktuell.
   die Hashtabelle `.h` hat 2n statt 2n+1 Plätze (Hashfunktion FNV-1a), und in `.i` steht der
   Schlüssel vor dem Offset statt dahinter. Alte /rdb-Indizes und `.h`-Dateien früherer
   Versionen dieses Nachbaus muss man neu bauen.
-- Summen, Mittelwerte und /act-Beträge sind Dezimalzahlen (`BigDecimal`), gerundet wird halb auf.
+- Summen, Mittelwerte und /act-Beträge sind Dezimalzahlen (`Rational`), gerundet wird halb auf.
 - Dateien, die kein UTF-8 sind, liest `rdb` als Latin-1 und schreibt sie als UTF-8. Fehler
   erscheinen als eine Zeile; mit `RDB_DEBUG=1` kommt der Ruby-Backtrace. Werte mit Tab oder
   Zeilenumbruch lehnen `append`, `enter`, `replace` und `ve` ab.

bin/rdb

@@ -1,8 +1,10 @@
-#!/usr/bin/env ruby
+#!/usr/bin/env -S ruby --disable-gems
 # frozen_string_literal: true
 
 # rdb -- every /rdb command is this one program, looking at the name it was
 # called by. `row`, `column`, `total` ... are symlinks to it (`rdb install`).
+# --disable-gems: rdb needs only the standard library, and skipping RubyGems
+# makes each command start about twice as fast.
 
 require_relative '../lib/rdb'
 

doc/review.md

@@ -81,22 +81,26 @@ Index ohne Entsprechung in der Suche offen.
 2. **Schlechte Hash-Funktion — Behoben** (FNV-1a, größter Eimer 4). Ursprünglich: ([fast.rb:68](../lib/rdb/fast.rb)):
    `bytes.sum % slots`. Bestätigt: 3000 fünfstellige Nummern landen in 30 Slots,
    größter Bucket 223. Abhilfe: FNV-1a o. Ä., Primzahl-Slotzahl.
-3. **Geld in `Float` — Behoben** (`BigDecimal` in `total`/`mean`/`subtotal` und /act). Ursprünglich: ([act.rb:11](../lib/rdb/cmd/act.rb), `to_f` im /act-Teil
+3. **Geld in `Float` — Behoben** (`Rational` in `total`/`mean`/`subtotal` und /act). Ursprünglich: ([act.rb:11](../lib/rdb/cmd/act.rb), `to_f` im /act-Teil
    und in `total`). Abhilfe: `BigDecimal` oder ganzzahlige Rappen.
-4. **awk-Brücke** ([env.rb:119-138](../lib/rdb/env.rb)): ganze Tabelle als String
+4. **awk-Brücke — bewusst offen.** `awk` und `awk_text` teilen sich jetzt den Aufruf. Streaming
+   und ein eigener Ausdrucksparser wären ein Umbau mit Verhaltensrisiko (awk-Syntax ist
+   dokumentierte Schnittstelle). Ursprünglich: ([env.rb:119-138](../lib/rdb/env.rb)): ganze Tabelle als String
    durch `capture3`, kein Streaming; handgeschriebener Parser erkennt Strings und
    Regexe heuristisch (`prev`) und hängt vom awk-Dialekt ab.
-5. **Globale Methoden auf `Object`** (`fast_opts`, `lock_file`, `num`, `money`,
+5. **Globale Methoden auf `Object` — Behoben** (`Rdb::Helpers`, Konstanten in `Rdb`). Ursprünglich: (`fast_opts`, `lock_file`, `num`, `money`,
    `ask`, `need`, `show_record` …). In ein Modul kapseln und in `Env` einbinden.
-6. **Doppelter Code.** `awk`/`awk_text`; Schlüsseltabellen-Parsing in `fast_opts`/
+6. **Doppelter Code — Behoben** (`Table.head_text`, `dash_cells`, `headed?`; `awk` über `awk_text`). Ursprünglich: `awk`/`awk_text`; Schlüsseltabellen-Parsing in `fast_opts`/
    `key_columns`; von Hand gebaute Kopf-/Strichzeilen trotz `Table.dash_row`.
-7. **Tests.** Laufzeit ca. 66 s wegen eines Ruby-Prozesses pro Test. Fehlerfälle
+7. **Tests — Behoben** (Suite 70 s → 21 s). Parallel, und `bin/rdb` startet mit `--disable-gems` (ca. 85 statt
+   170 ms je Befehl). Fehlerfälle haben jetzt Tests. Den Rest der Zeit braucht
+   `test_testall_runs_demo_script` (rund 300 Befehle nacheinander). Ursprünglich: Laufzeit ca. 66 s wegen eines Ruby-Prozesses pro Test. Fehlerfälle
    (Encoding, Tab in Wert, Locks, Editor) nicht abgedeckt; Tests direkt gegen
    `Rdb.run` im Prozess wären schneller.
-8. **Repo.** `demo/fact.pl`, `fact_lc.pl`, `rule.pl` sind ungetrackt;
+8. **Repo — offen, Entscheidung beim Autor.** Ursprünglich: `demo/fact.pl`, `fact_lc.pl`, `rule.pl` sind ungetrackt;
    `demo/testall.old` (1187 Zeilen) wäre als Golden-File-Test besser gepflegt.
 
 ## Empfohlene Reihenfolge
 
-Schwachstellen, Fehler und Verbesserung 1–3 sind erledigt. Offen sind 4–8: die
-awk-Brücke, globale Methoden auf `Object`, doppelter Code, Testlaufzeit, Repo-Aufräumen.
+Schwachstellen, Fehler und Verbesserungen 1–3 und 5–7 sind erledigt.
+Offen: die awk-Brücke (4, bewusst) und das Repo-Aufräumen (8).

lib/rdb.rb

@@ -3,7 +3,6 @@
 # A Ruby re-creation of /rdb, the relational database tools Mark Williams
 # Company sold for COHERENT (~1991).
 
-require 'bigdecimal'
 require 'open3'
 require 'date'
 require 'tmpdir'
@@ -33,6 +32,25 @@ module Rdb
       hits.first
     end
 
+    # Exact decimal arithmetic with Rational (core Ruby, so bin/rdb can start
+    # without RubyGems): 0.1 + 0.2 is 0.3 and money adds up to the cent.
+    def decimal(str)
+      s = str.to_s.strip
+      return 0r if s.empty?
+
+      Rational(s.match?(NUM) ? s.sub(/\.\z/, '') : s.to_f.to_s)
+    end
+
+    # num with dec places, rounded half up ("%.2f" rounds the binary value).
+    def fixed(num, dec)
+      r = num.is_a?(Float) ? Rational(num.to_s) : num.to_r
+      r = r.round(dec, half: :up)
+      return r.to_i.to_s if dec.zero?
+
+      digits = (r.abs * 10**dec).to_i.to_s.rjust(dec + 1, '0')
+      "#{'-' if r.negative?}#{digits[0...-dec]}.#{digits[-dec..]}"
+    end
+
     def die(msg)
       abort "#{File.basename($PROGRAM_NAME)}: #{msg}"
     end

lib/rdb/cmd/access.rb

@@ -6,6 +6,8 @@
 require 'stringio'
 require_relative '../fast'
 
+module Rdb
+  module Helpers
     # Parse the common fast-access options. Stops at the first non-option, which
     # is left in args; returns the option hash.
     def fast_opts(args)
@@ -41,8 +43,7 @@ def key_columns(args, data)
       lines = text.lines.map { |l| l.chomp("\n") }
       cols = args
       if cols.empty?
-    keyed = lines.size >= 2 && Rdb::Table.dash_line?(lines[1].split("\t", -1))
-    cols = keyed ? lines[0].split("\t", -1).map(&:strip) : [data.head[0]]
+        cols = Rdb::Table.headed?(lines) ? lines[0].split("\t", -1).map(&:strip) : [data.head[0]]
       end
       [cols, Rdb::Fast.read_keys(StringIO.new(text), cols.size)]
     end
@@ -56,6 +57,8 @@ def each_key(ncols, keys = nil, &block)
         Rdb::Fast.read_keys($stdin, ncols).each { |k| yield k }
       end
     end
+  end
+end
 
 Rdb.register('index', 'Set up table for search', 'index [-m[bhirs]] [-x] [-l] tableorlist [keycol ...]') do |args|
   o = fast_opts(args)
@@ -76,7 +79,7 @@ Rdb.register('search', 'Search a table',
   data = Rdb::Fast::Data.new(table)
   need_index(table, o[:m])
   keycols, keys = key_columns(args, data)
-  puts data.head.join("\t"), Rdb::Table.dash_row(data.head) unless o[:h]
+  $stdout.write Rdb::Table.head_text(data.head) unless o[:h]
   each_key(keycols.size, keys) do |key|
     hits = Rdb::Fast.search(data, o[:m], keycols, key, first: o[:s] || o[:l], partial: o[:x], exact: o[:exact])
     hits.each do |h|
@@ -97,7 +100,7 @@ Rdb.register('seek', 'Return the beginning and ending offset of a row', 'seek [-
     Rdb::Fast.search(data, o[:m], keycols, key, first: true).each do |h|
       puts "#{h.row.from} #{h.row.to} #{h.xfrom} #{h.xto}"
       if o[:o]
-        File.write(o[:o], data.head.join("\t") + "\n" + Rdb::Table.dash_row(data.head) + "\n" + h.row.line + "\n")
+        File.write(o[:o], Rdb::Table.head_text(data.head) + h.row.line + "\n")
       end
     end
   end
@@ -137,7 +140,7 @@ Rdb.register('replace', 'Insert a record into a file at specified location',
 
   text = $stdin.read
   lines = text.lines.map { |l| l.chomp("\n") }
-  lines = lines.drop(2) if lines.size >= 2 && Rdb::Table.dash_line?(lines[1].split("\t", -1))
+  lines = lines.drop(2) if Rdb::Table.headed?(lines)
   die 'replace: no input row' if lines.empty?
 
   keycol = Rdb::Fast::Data.new(table).head[0]
@@ -189,11 +192,17 @@ require 'shellwords'
 
 # ---- locks -------------------------------------------------------------------
 
+module Rdb
+  module Helpers
     # Lock files live in a private per-user directory, not loose in the shared
     # /tmp, so other users can neither forge nor delete them.
     def lock_dir
       dir = File.join(Dir.tmpdir, "rdb-locks-#{Process.uid}")
-  Dir.mkdir(dir, 0o700) unless File.directory?(dir)
+      begin
+        Dir.mkdir(dir, 0o700)
+      rescue Errno::EEXIST
+        nil # made by someone else, maybe at this very moment; checked below
+      end
       st = File.lstat(dir)
       Rdb.die("#{dir}: not a private directory of yours") if st.symlink? || st.uid != Process.uid || (st.mode & 0o077) != 0
       dir
@@ -218,6 +227,8 @@ def with_lock_file(name)
         f.write(lines.map { |l| "#{l}\n" }.join)
       end
     end
+  end
+end
 
 Rdb.register('lock', 'Lock a record or field of a file', 'lock filename processid from to indexfrom indexto') do |args|
   die 'usage: lock filename processid from to indexfrom indexto' if args.size < 4
@@ -267,6 +278,8 @@ end
 
 # ---- interactive editing --------------------------------------------------------
 
+module Rdb
+  module Helpers
     def show_record(data_head, cells)
       data_head.each_with_index { |h, i| puts "#{i + 1}  #{h}\t#{cells[i].to_s.strip}" }
     end
@@ -283,6 +296,8 @@ def edit_cells(head, cells)
         head.map { |h| (j = t.head.index(h)) ? t.rows[0][j] : '' }
       end
     end
+  end
+end
 
 Rdb.register('update.rdb', 'Display and edit records in any sized file',
              'update.rdb [-l -m[bhirs] -v] tableorlist [keycolumn ... ]') do |args|
@@ -443,7 +458,7 @@ end
 Rdb.register('validate', 'Find invalid data', "validate 'pattern { action } ...' < tableorlist") do |args|
   die "usage: validate 'pattern { action } ...' < table" unless args.size == 1
   t = table_in
-  puts t.head.join("\t"), Rdb::Table.dash_row(t.head)
+  $stdout.write Rdb::Table.head_text(t.head)
   prog = "BEGIN{OFS=\"\\t\"} #{to_awk(args[0], t.head)} {print}"
   out, ok = awk_text(prog, t.rows)
   $stdout.write out
@@ -512,6 +527,8 @@ end
 
 # ---- exercising the fast-access methods ---------------------------------------------
 
+module Rdb
+  module Helpers
     def sample_table(path, n = 2000)
       rows = (1..n).map { |i| [i.to_s, "name#{i * 7919 % n}", "city#{i % 97}"] }
       File.write(path, Rdb::Table.new(%w[Id Name City], rows).to_s)
@@ -536,6 +553,8 @@ def time_methods(n = 2000)
         end
       end
     end
+  end
+end
 
 Rdb.register('timesearch', 'Time the fast-access methods', 'timesearch [ > timesearch.new ]') do |_args|
   puts 'method  index(s)  200 searches(s)  rows found'

lib/rdb/cmd/act.rb

@@ -5,26 +5,23 @@
 # an accounting package: each command reads and writes plain /rdb tables in
 # the current directory.
 
-ACT_COMMANDS = %w[act adjust balance bom calcpay calculate cashflow chartdup close consolidate cstate explode
+Rdb::ACT_COMMANDS = %w[act adjust balance bom calcpay calculate cashflow chartdup close consolidate cstate explode
                   fillform foot getjournal invoice sale tax update.inv w2].freeze
 
-# Amounts are decimal numbers, not floats: sums of cents stay exact.
+module Rdb
+  module Helpers
+    # Amounts are exact decimals (Rational), not floats: sums of cents stay exact.
     def num(str)
-  s = str.to_s.strip
-  s.match?(Rdb::NUM) ? BigDecimal(s.sub(/\.\z/, '')) : BigDecimal(s.to_f.to_s)
+      Rdb.decimal(str)
     end
 
     # Two decimals, always (money drops them from whole amounts).
     def fixed2(val)
-  val = BigDecimal(val.to_s) unless val.is_a?(BigDecimal)
-  val.round(2, :half_up).to_s('F').then { |t| t.sub(/\.(\d*)\z/) { ".#{Regexp.last_match(1).ljust(2, '0')}" } }
+      Rdb.fixed(val, 2)
     end
 
     def money(val)
-  val = BigDecimal(val.to_s) unless val.is_a?(BigDecimal)
-  return val.to_i.to_s if val == val.to_i
-
-  val.round(2, :half_up).to_s('F').sub(/\.(\d)\z/, '.\10')
+      val == val.to_i ? val.to_i.to_s : Rdb.fixed(val, 2)
     end
 
     def need(file)
@@ -39,9 +36,11 @@ end
     def account_key(acct)
       [acct.to_f, acct]
     end
+  end
+end
 
 Rdb.register('act', 'List all /act commands', 'act') do |_args|
-  names = ACT_COMMANDS.sort
+  names = Rdb::ACT_COMMANDS.sort
   w = names.map(&:size).max + 2
   names.each_slice(5) { |row| puts row.map { |n| n.ljust(w) }.join.rstrip }
 end
@@ -66,7 +65,7 @@ Rdb.register('foot', 'Foot or subtotal Debits and Credit of ledger', 'foot') do
   out = []
   t.rows.sort_by { |r| account_key(r[0]) }.group_by { |r| r[0] }.each_value do |rows|
     out.concat(rows)
-    out << t.head.map { |h| '-' * [h.size, 1].max }
+    out << Rdb::Table.dash_cells(t.head)
     sub = [''] * t.head.size
     sub[0] = rows.first[0]
     sub[d] = money(rows.sum { |r| num(r[d]) })
@@ -84,7 +83,7 @@ Rdb.register('adjust', 'Create adjusted trial balance table', 'adjust') do |_arg
     m = (meta[h['Account']] ||= {})
     h.each { |k, v| m[k] = v if !v.to_s.strip.empty? && !%w[Debit Credit].include?(k) && !m.key?(k) }
   end
-  net = Hash.new(BigDecimal(0))
+  net = Hash.new(0r)
   [tb, ja].each do |t|
     d = t.index_of('Debit')
     c = t.index_of('Credit')
@@ -104,12 +103,16 @@ Rdb.register('adjust', 'Create adjusted trial balance table', 'adjust') do |_arg
   File.write('adjustedtrial', Rdb::Table.new(head, rows).to_s)
 end
 
+module Rdb
+  module Helpers
     def trial_parts(t)
       h = t.head
       fmt = h.index('Format')
       rows = t.rows.map { |r| { acct: r[0], debit: num(r[h.index('Debit')]), credit: num(r[h.index('Credit')]), fmt: r[fmt].to_s, name: r[h.index('Name')].to_s, row: r } }
       rows
     end
+  end
+end
 
 Rdb.register('balance', 'Create balance sheet from adjusted trial balance', 'balance') do |_args|
   rows = trial_parts(need('adjustedtrial'))
@@ -183,7 +186,7 @@ Rdb.register('fillform', 'Fill a tax from with adjusted trial balance data', 'fi
   lines = need("taxform.#{args[0]}")
   trial = need('adjustedtrial')
   ti = trial.index_of('Taxline')
-  sums = Hash.new(BigDecimal(0))
+  sums = Hash.new(0r)
   trial.rows.each { |r| sums[r[ti]] += (num(r[trial.index_of('Debit')]) - num(r[trial.index_of('Credit')])).abs }
   rows = lines.rows.map { |r| [r[0], money(sums[r[0]]), r[1].to_s] }
   File.write(args[0], Rdb::Table.new(%w[Taxline Amount Description], rows).to_s)
@@ -205,14 +208,14 @@ Rdb.register('tax', 'Compute tax from income and tax table', 'tax income < taxta
   income = num(args[0])
   rows = table_in.rows.map { |r| r.map { |c| num(c) } }.sort_by(&:first)
   row = rows.reverse.find { |r| r[0] <= income }
-  puts format('%7d', row ? (row[1] + (income - row[0]) * row[2] / 100.0).round : 0)
+  puts format('%7d', row ? (row[1] + (income - row[0]) * row[2] / 100).round(half: :up) : 0)
 end
 
 Rdb.register('cashflow', 'Compute balance column of cash table', 'cashflow < cashtable') do |_args|
   t = table_in
   a = t.index_of('Amount')
   b = t.index_of('Balance')
-  bal = BigDecimal(0)
+  bal = 0r
   rows = t.rows.map do |r|
     r = r.dup
     if r[a].strip.empty? && !r[b].strip.empty?
@@ -250,9 +253,9 @@ end
 Rdb.register('bom', 'Produce bill-of-materials from parts list', 'bom') do |_args|
   part = need('part')
   items = need('saleitem')
-  qty = Hash.new(BigDecimal(0))
+  qty = Hash.new(0r)
   items.rows.each { |r| qty[r[items.index_of('Code')]] += num(r[items.index_of('Qty')]) }
-  need_total = Hash.new(BigDecimal(0))
+  need_total = Hash.new(0r)
   part.rows.each do |r|
     code = r[part.index_of('Code')]
     need_total[r[part.index_of('Subpart')]] += qty[code] * num(r[part.index_of('Count')]) if qty.key?(code)
@@ -263,7 +266,7 @@ end
 Rdb.register('explode', 'Produce table of subparts and their count for a part', 'explode Part Amount [ table ]') do |args|
   die 'usage: explode Part Amount [table]' if args.size < 2
   t = need(args[2] || 'parttable')
-  totals = Hash.new(BigDecimal(0))
+  totals = Hash.new(0r)
   queue = [[args[0], num(args[1])]]
   until queue.empty?
     part, amount = queue.shift
@@ -275,6 +278,8 @@ end
 
 # ---- sales ------------------------------------------------------------------------------
 
+module Rdb
+  module Helpers
     def ask(prompt)
       print prompt
       $stdout.flush
@@ -290,6 +295,8 @@ def address_block(customers, number)
        [h['Title'], h['First'], h['Last'], h['Position']].compact.join(' ').strip,
        h['Company'], h['Street'], [h['City'], h['State'], h['Country']].compact.join(' ')].compact.reject(&:empty?)
     end
+  end
+end
 
 Rdb.register('invoice', 'Print invoice for a sale order', 'invoice') do |_args|
   orders = need('salesorder')
@@ -359,7 +366,7 @@ Rdb.register('sale', 'Enter sale order, and item, update customer', 'sale') do |
     end
     puts address_block(customers, cust)
     n = Rdb::Table.open('salesorder').rows.map { |r| r[0].to_i }.max.to_i + 1
-    gross = BigDecimal(0)
+    gross = 0r
     items = []
     loop do
       code = ask('Code (Return when done): ')
@@ -375,7 +382,7 @@ Rdb.register('sale', 'Enter sale order, and item, update customer', 'sale') do |
     end
     next if items.empty?
 
-    tax = gross * BigDecimal('0.05')
+    tax = gross * Rational('0.05')
     order = [n.to_s, cust, next_date, '0', fixed2(gross), fixed2(tax), fixed2(gross + tax)]
     puts Rdb::Table.new(%w[Number Cust Date Shipped Gross Tax Total], [order]).to_s
     if ask('Is this correct? (y, n) ').downcase.start_with?('y')

lib/rdb/cmd/convert.rb

@@ -105,6 +105,8 @@ end
 
 # ---- names that sound alike ----------------------------------------------------
 
+module Rdb
+  module Helpers
     def soundex_code(word)
       codes = { 'BFPV' => '1', 'CGJKQSXZ' => '2', 'DT' => '3', 'L' => '4', 'MN' => '5', 'R' => '6' }
       letters = word.upcase.gsub(/[^A-Z]/, '').chars
@@ -121,6 +123,8 @@ def soundex_code(word)
       end
       out.ljust(4, '0')
     end
+  end
+end
 
 Rdb.register('soundex', 'Convert a name into soundex code', 'soundex < file > file.x ; index -mb file.x Soundex') do |_args|
   t = table_in

lib/rdb/cmd/relational.rb

@@ -189,7 +189,7 @@ end
 # total, mean, maximum, minimum, width, precision and datatype are one program
 # that decides what to compute from the name it was called by.
 
-STATS = {
+Rdb::STATS = {
   'total' => 'Sum a column',
   'mean' => 'Display the mean of a column',
   'maximum' => 'Display the maximum value in a column',
@@ -199,7 +199,7 @@ STATS = {
   'datatype' => 'Display the data type of each column selected'
 }.freeze
 
-STATS.each do |cmd, desc|
+Rdb::STATS.each do |cmd, desc|
   Rdb.register(cmd, desc, "#{cmd} [ -l ] [ column ... ] < table") do |args|
     list = args.first == '-l'
     args.shift if list
@@ -209,7 +209,7 @@ STATS.each do |cmd, desc|
     sel.each do |i|
       vals = t.rows.map { |r| r[i] }
       filled = vals.reject { |v| v.strip.empty? }
-      nums = filled.all? { |v| numeric?(v) } ? vals.map { |v| v.strip.empty? ? BigDecimal(0) : dec_num(v) } : nil
+      nums = filled.all? { |v| numeric?(v) } ? vals.map { |v| dec_num(v) } : nil
       dec = filled.map { |v| decimals(v) }.max.to_i
       result[i] =
         case name
@@ -260,7 +260,7 @@ Rdb.register('subtotal', 'Output subtotals of columns in a table',
     end
     if list
       out.concat(group)
-      out << t.head.map { |h| '-' * [h.size, 1].max }
+      out << Rdb::Table.dash_cells(t.head)
     end
     out << sub
     group = []

lib/rdb/cmd/report.rb

@@ -4,6 +4,8 @@
 
 require 'shellwords'
 
+module Rdb
+  module Helpers
     # Fill in a form for one row: <Column> is replaced by the value, <!command!>
     # by the output of the shell command, and <!cmd <Column>!> passes data to it.
     # Values inside a command are shell-escaped, so a cell can never add commands
@@ -20,6 +22,8 @@ def fill_form(form, head, row)
         end
       end
     end
+  end
+end
 
 Rdb.register('report', 'Write reports using a form and a table', 'report form < tableorlist') do |args|
   die 'usage: report form < tableorlist' unless args.size == 1
@@ -76,10 +80,14 @@ Rdb.register('screen', 'Convert a form into a screen-input shell program', 'scre
   puts ': append row', %(echo "#{names.map { |x| "$#{x}" }.join("\t")}")
 end
 
+module Rdb
+  module Helpers
     def label_lines(head, row)
       h = head.zip(row).to_h
       [h['Name'], h['Company'], h['Street'], "#{h['City']}, #{h['State']} #{h['ZIP']}".strip].compact
     end
+  end
+end
 
 Rdb.register('label', 'Print mailing labels from a mailing list', 'label < list') do |_args|
   t = table_in

lib/rdb/cmd/term.rb

@@ -4,14 +4,16 @@ require 'shellwords'
 
 # Help, manual pages, terminal helpers, menus and housekeeping.
 
-ESC = "\e"
-TERM_CAPS = {
-  'cl' => "#{ESC}[H#{ESC}[2J", 'ho' => "#{ESC}[H", 'ce' => "#{ESC}[K", 'cd' => "#{ESC}[J",
-  'cm' => "#{ESC}[%i%d;%dH", 'so' => "#{ESC}[7m", 'se' => "#{ESC}[m", 'us' => "#{ESC}[4m",
-  'ue' => "#{ESC}[m", 'mb' => "#{ESC}[5m", 'md' => "#{ESC}[1m", 'me' => "#{ESC}[m",
-  'as' => "#{ESC}(0", 'ae' => "#{ESC}(B", 'up' => "#{ESC}[A", 'do' => "#{ESC}[B", 'bs' => "\b"
+Rdb::ESC = "\e"
+Rdb::TERM_CAPS = {
+  'cl' => "#{Rdb::ESC}[H#{Rdb::ESC}[2J", 'ho' => "#{Rdb::ESC}[H", 'ce' => "#{Rdb::ESC}[K", 'cd' => "#{Rdb::ESC}[J",
+  'cm' => "#{Rdb::ESC}[%i%d;%dH", 'so' => "#{Rdb::ESC}[7m", 'se' => "#{Rdb::ESC}[m", 'us' => "#{Rdb::ESC}[4m",
+  'ue' => "#{Rdb::ESC}[m", 'mb' => "#{Rdb::ESC}[5m", 'md' => "#{Rdb::ESC}[1m", 'me' => "#{Rdb::ESC}[m",
+  'as' => "#{Rdb::ESC}(0", 'ae' => "#{Rdb::ESC}(B", 'up' => "#{Rdb::ESC}[A", 'do' => "#{Rdb::ESC}[B", 'bs' => "\b"
 }.freeze
 
+module Rdb
+  module Helpers
     def command_rows
       Rdb::COMMANDS.values.reject { |c| c.name.include?('.list') || c.name == 'rdb.install' }.sort_by(&:name)
     end
@@ -19,6 +21,8 @@ end
     def syntax_of(cmd)
       cmd.syntax.sub(/\A\S+\s*/, '')
     end
+  end
+end
 
 Rdb.register('rdb.list', 'List all /rdb programs', 'rdb') do |_args|
   names = command_rows.map(&:name)
@@ -80,6 +84,8 @@ end
 
 # ---- manual pages ---------------------------------------------------------------
 
+module Rdb
+  module Helpers
     def troff_clean(str)
       # a literal backslash (\\ or \e) is parked in \0 so that later rules cannot see it
       str.gsub('\\\\', "\0").gsub(/\\f[BIRP]/, '').gsub('\\e', "\0").gsub('\\(em', ' -- ').gsub('\\(aa', "'")
@@ -131,6 +137,8 @@ def render_man(src)
       flush.()
       out.join("\n").squeeze("\n")
     end
+  end
+end
 
 Rdb.register('man', 'Display the manual page for a command', 'man [-g] command') do |args|
   groff = args.first == '-g'
@@ -161,21 +169,21 @@ end
 
 Rdb.register('termput', 'Get terminal capability from /etc/termcap file', 'termput capability') do |args|
   die 'usage: termput capability' unless args.size == 1
-  $stdout.print TERM_CAPS.fetch(args[0]) { die "no such capability '#{args[0]}'" }
+  $stdout.print Rdb::TERM_CAPS.fetch(args[0]) { die "no such capability '#{args[0]}'" }
 end
 
 Rdb.register('tset', 'Fetch termcap entry for a terminal type', 'tset [ TERM ]') do |args|
   term = args.first || ENV['TERM'] || 'ansi'
-  puts ":#{TERM_CAPS.map { |k, v| "#{k}=#{v.inspect[1..-2]}" }.join(':')}:co##{ENV['COLUMNS'] || 80}:li##{ENV['LINES'] || 24}:tc=#{term}:"
+  puts ":#{Rdb::TERM_CAPS.map { |k, v| "#{k}=#{v.inspect[1..-2]}" }.join(':')}:co##{ENV['COLUMNS'] || 80}:li##{ENV['LINES'] || 24}:tc=#{term}:"
 end
 
 Rdb.register('clear.rdb', "Clear the terminal's screen", 'clear.rdb') do |_args|
-  $stdout.print TERM_CAPS['cl']
+  $stdout.print Rdb::TERM_CAPS['cl']
 end
 
 Rdb.register('cursor', 'Move the cursor to the row and column requested', 'cursor row col [ CURSOR ]') do |args|
   die 'usage: cursor row col [CURSOR]' if args.size < 2
-  $stdout.print "#{ESC}[#{args[0].to_i};#{args[1].to_i}H"
+  $stdout.print "#{Rdb::ESC}[#{args[0].to_i};#{args[1].to_i}H"
   $stdout.flush
 end
 

lib/rdb/cmd/text.rb

@@ -2,6 +2,8 @@
 
 # Formatting tables and working with text.
 
+module Rdb
+  module Helpers
     def justify_numbers(cells)
       parts = cells.map { |c| c.empty? ? ['', nil] : c.split('.', 2) }
       int_w = parts.map { |i, _| i.size }.max
@@ -14,6 +16,8 @@ def justify_numbers(cells)
         s
       end
     end
+  end
+end
 
 Rdb.register('justify', 'Left or right justify the columns of a table', "justify [-elrct'c'] [ column ... ] < table") do |args|
   expand = false
@@ -112,7 +116,7 @@ Rdb.register('pad', 'Add extra spaces at end of last column', 'pad [-number] < t
   if t.list
     t.with(rows: t.rows.map { |r| r[0...-1] + [r.last + ' ' * n] }).write
   else
-    $stdout.write t.head.join("\t"), "\n", Rdb::Table.dash_row(t.head), "\n"
+    $stdout.write Rdb::Table.head_text(t.head)
     t.rows.each { |r| puts (r[0...-1] + [r.last + ' ' * n]).join("\t") }
   end
 end

lib/rdb/env.rb

@@ -1,8 +1,13 @@
 # frozen_string_literal: true
 
 module Rdb
+  # Helpers of single command files (lib/rdb/cmd/*.rb) go in here, not on
+  # Object; every Env includes them.
+  module Helpers; end
+
   # Command blocks run inside an Env, which holds the helpers they share.
   class Env
+    include Helpers
     attr_reader :name
 
     def initialize(name)
@@ -30,14 +35,12 @@ module Rdb
       str.strip[/\.(\d+)\z/, 1].to_s.size
     end
 
-    # Decimal arithmetic, so that 0.1 + 0.2 is 0.3 and money adds up to the cent.
     def dec_num(str)
-      BigDecimal(str.strip.sub(/\.\z/, ''))
+      Rdb.decimal(str)
     end
 
     def fmt(num, dec)
-      num = BigDecimal(num.to_s) unless num.is_a?(BigDecimal)
-      dec.zero? ? num.round(0, :half_up).to_i.to_s : num.round(dec, :half_up).to_s('F').then { |t| t.sub(/\.(\d*)\z/) { ".#{Regexp.last_match(1).ljust(dec, '0')}" } }
+      Rdb.fixed(num, dec)
     end
 
     def blank_row?(row)
@@ -102,18 +105,14 @@ module Rdb
 
     # Run an awk program over rows (tab separated) and return the output rows.
     def awk(prog, rows)
-      input = rows.map { |r| r.join("\t") }.join("\n")
-      input += "\n" unless input.empty?
-      out, err, st = Open3.capture3('awk', '-F', "\t", prog, stdin_data: input)
-      $stderr.print err unless err.empty?
-      exit 1 unless st.success?
+      out, ok = awk_text(prog, rows)
+      exit 1 unless ok
       out.each_line.map { |l| l.chomp.split("\t", -1) }
     end
 
     # Raw output of awk, for programs that print their own text.
     def awk_text(prog, rows)
-      input = rows.map { |r| r.join("\t") }.join("\n")
-      input += "\n" unless input.empty?
+      input = rows.map { |r| "#{r.join("\t")}\n" }.join
       out, err, st = Open3.capture3('awk', '-F', "\t", prog, stdin_data: input)
       $stderr.print err unless err.empty?
       [out, st.success?]

lib/rdb/fast.rb

@@ -129,7 +129,7 @@ module Rdb
       when 'b'
         k = data.column(keycol)
         body = live_rows(data).sort_by { |r| r.cells[k].to_s.strip }.map { |r| r.line + "\n" }
-        Rdb.write_file(path, data.head.join("\t") + "\n" + Table.dash_row(data.head) + "\n" + body.join)
+        Rdb.write_file(path, Table.head_text(data.head) + body.join)
       end
     end
 
@@ -393,7 +393,7 @@ module Rdb
       return [] if text.empty?
 
       lines = text.lines.map { |l| l.chomp("\n") }
-      if lines.size >= 2 && Table.dash_line?(lines[1].split("\t", -1))
+      if Table.headed?(lines)
         lines.drop(2).reject(&:empty?).map { |l| l.split("\t", -1).first(ncols) }
       else
         lines.reject(&:empty?).map { |l| l.split("\t", -1).first(ncols) }

lib/rdb/table.rb

@@ -28,6 +28,11 @@ module Rdb
       end
     end
 
+    # Do these lines (chomped) start with a head line and a dash line?
+    def self.headed?(lines)
+      lines.size >= 2 && dash_line?(lines[1].split("\t", -1))
+    end
+
     def self.parse(text)
       text = utf8(text)
       return parse_list(text) if text.start_with?("\n")
@@ -78,15 +83,22 @@ module Rdb
     def to_s
       return list_text if @list
 
-      out = +''
-      out << @head.join("\t") << "\n"
-      out << Table.dash_row(@head) << "\n"
+      out = Table.head_text(@head)
       @rows.each { |r| out << r.join("\t") << "\n" }
       out
     end
 
+    def self.dash_cells(head)
+      head.map { |h| '-' * [h.size, 1].max }
+    end
+
     def self.dash_row(head)
-      head.map { |h| '-' * [h.size, 1].max }.join("\t")
+      dash_cells(head).join("\t")
+    end
+
+    # Head line and dash line, each with its newline.
+    def self.head_text(head)
+      +"#{head.join("\t")}\n#{dash_row(head)}\n"
     end
 
     def list_text

test/test_helper.rb

@@ -12,6 +12,10 @@ DEMO = File.join(ROOT, 'demo')
 # Each test runs in a scratch copy of demo/, so commands that change files
 # (append, delete, replace ...) never touch the originals.
 class RdbCase < Minitest::Test
+  # Tests spawn their own processes in their own directory, so they can run
+  # side by side (N=1 in MT_CPU runs them one at a time).
+  parallelize_me!
+
   def setup
     @dir = Dir.mktmpdir('rdb-test')
     FileUtils.cp_r(Dir[File.join(DEMO, '*')], @dir)