Schneller Start ohne RubyGems, Rational statt BigDecimal, Helfer in Rdb::Helpers

bin/rdb läuft mit --disable-gems (ca. 85 statt 170 ms je Befehl), dafür
rechnen Summen und /act mit Rational. Hilfsfunktionen und Konstanten liegen
nicht mehr auf Object, Kopf-/Strichzeile und awk-Aufruf an einer Stelle.
Tests laufen parallel: 21 statt 70 Sekunden.
This commit is contained in:
wkh
2026-10-30 10:01:36 +01:00
parent 1c489f6994
commit c2ea36c6d6
16 changed files with 380 additions and 287 deletions
+5 -1
View File
@@ -20,7 +20,11 @@ folgen [Semantic Versioning](https://semver.org/lang/de/). Die Version steht in
### Verbessert ### Verbessert
- Schnellzugriff: `search`, `seek`, `delete` und `update.rdb` lesen bei `-mh`, `-mi` und `-mr` nur noch den Indexeintrag und den einen Satz (Seek auf Byte-Positionen, bei `.i` Halbierung über die Datei), nicht mehr Tabelle und Index vollständig. `Data` liest die Zeilen erst, wenn eine Methode alle braucht. - Schnellzugriff: `search`, `seek`, `delete` und `update.rdb` lesen bei `-mh`, `-mi` und `-mr` nur noch den Indexeintrag und den einen Satz (Seek auf Byte-Positionen, bei `.i` Halbierung über die Datei), nicht mehr Tabelle und Index vollständig. `Data` liest die Zeilen erst, wenn eine Methode alle braucht.
- Die Hashfunktion ist FNV-1a statt der Bytesumme. Vorher fielen 3000 fünfstellige Nummern in 30 Plätze (größter Eimer 223), jetzt liegt der größte Eimer bei etwa 4. **Hashindizes (`.h`) müssen neu gebaut werden** (`index -mh`). - Die Hashfunktion ist FNV-1a statt der Bytesumme. Vorher fielen 3000 fünfstellige Nummern in 30 Plätze (größter Eimer 223), jetzt liegt der größte Eimer bei etwa 4. **Hashindizes (`.h`) müssen neu gebaut werden** (`index -mh`).
- `total`, `mean`, `subtotal` und die /act-Befehle rechnen mit `BigDecimal` statt `Float`: `0.1 + 0.2 + 0.7` ergibt `1.0`, Beträge summieren sich auf den Rappen genau, und es wird kaufmännisch gerundet (halb auf). - `total`, `mean`, `subtotal` und die /act-Befehle rechnen exakt mit `Rational` statt `Float`: `0.1 + 0.2 + 0.7` ergibt `1.0`, Beträge summieren sich auf den Rappen genau, und es wird kaufmännisch gerundet (halb auf).
- `bin/rdb` startet mit `ruby --disable-gems`: jeder Befehl ist in etwa doppelt so schnell gestartet (ca. 85 statt 170 ms), weil RubyGems nicht geladen wird. `rdb` braucht nur die Standardbibliothek.
- Die Tests laufen parallel (`parallelize_me!`; `MT_CPU=1` für nacheinander). Zusammen mit dem schnelleren Start dauert die Suite 21 statt 70 Sekunden.
- Intern: Hilfsfunktionen der Befehlsdateien liegen in `Rdb::Helpers` statt auf `Object`, die Konstanten (`STATS`, `TERM_CAPS` ...) in `Rdb`. Kopf- und Strichzeile entstehen an einer Stelle (`Table.head_text`, `dash_cells`, `headed?`), `awk` baut auf `awk_text` auf.
### Sicherheit ### Sicherheit
- `report`: Spaltenwerte in `<!befehl <Spalte>!>` wurden unmaskiert in einen Shell-Aufruf eingesetzt, eine Zelle konnte so Befehle ausführen. Sie werden jetzt mit `Shellwords.escape` eingesetzt. - `report`: Spaltenwerte in `<!befehl <Spalte>!>` wurden unmaskiert in einen Shell-Aufruf eingesetzt, eine Zelle konnte so Befehle ausführen. Sie werden jetzt mit `Shellwords.escape` eingesetzt.
+1 -1
View File
@@ -246,7 +246,7 @@ er nicht mehr passt (außer mit `-a`). Beide halten die Indizes aktuell.
die Hashtabelle `.h` hat 2n statt 2n+1 Plätze (Hashfunktion FNV-1a), und in `.i` steht der die Hashtabelle `.h` hat 2n statt 2n+1 Plätze (Hashfunktion FNV-1a), und in `.i` steht der
Schlüssel vor dem Offset statt dahinter. Alte /rdb-Indizes und `.h`-Dateien früherer Schlüssel vor dem Offset statt dahinter. Alte /rdb-Indizes und `.h`-Dateien früherer
Versionen dieses Nachbaus muss man neu bauen. Versionen dieses Nachbaus muss man neu bauen.
- Summen, Mittelwerte und /act-Beträge sind Dezimalzahlen (`BigDecimal`), gerundet wird halb auf. - Summen, Mittelwerte und /act-Beträge sind Dezimalzahlen (`Rational`), gerundet wird halb auf.
- Dateien, die kein UTF-8 sind, liest `rdb` als Latin-1 und schreibt sie als UTF-8. Fehler - Dateien, die kein UTF-8 sind, liest `rdb` als Latin-1 und schreibt sie als UTF-8. Fehler
erscheinen als eine Zeile; mit `RDB_DEBUG=1` kommt der Ruby-Backtrace. Werte mit Tab oder erscheinen als eine Zeile; mit `RDB_DEBUG=1` kommt der Ruby-Backtrace. Werte mit Tab oder
Zeilenumbruch lehnen `append`, `enter`, `replace` und `ve` ab. Zeilenumbruch lehnen `append`, `enter`, `replace` und `ve` ab.
+3 -1
View File
@@ -1,8 +1,10 @@
#!/usr/bin/env ruby #!/usr/bin/env -S ruby --disable-gems
# frozen_string_literal: true # frozen_string_literal: true
# rdb -- every /rdb command is this one program, looking at the name it was # rdb -- every /rdb command is this one program, looking at the name it was
# called by. `row`, `column`, `total` ... are symlinks to it (`rdb install`). # called by. `row`, `column`, `total` ... are symlinks to it (`rdb install`).
# --disable-gems: rdb needs only the standard library, and skipping RubyGems
# makes each command start about twice as fast.
require_relative '../lib/rdb' require_relative '../lib/rdb'
+12 -8
View File
@@ -81,22 +81,26 @@ Index ohne Entsprechung in der Suche offen.
2. **Schlechte Hash-Funktion — Behoben** (FNV-1a, größter Eimer 4). Ursprünglich: ([fast.rb:68](../lib/rdb/fast.rb)): 2. **Schlechte Hash-Funktion — Behoben** (FNV-1a, größter Eimer 4). Ursprünglich: ([fast.rb:68](../lib/rdb/fast.rb)):
`bytes.sum % slots`. Bestätigt: 3000 fünfstellige Nummern landen in 30 Slots, `bytes.sum % slots`. Bestätigt: 3000 fünfstellige Nummern landen in 30 Slots,
größter Bucket 223. Abhilfe: FNV-1a o. Ä., Primzahl-Slotzahl. größter Bucket 223. Abhilfe: FNV-1a o. Ä., Primzahl-Slotzahl.
3. **Geld in `Float` — Behoben** (`BigDecimal` in `total`/`mean`/`subtotal` und /act). Ursprünglich: ([act.rb:11](../lib/rdb/cmd/act.rb), `to_f` im /act-Teil 3. **Geld in `Float` — Behoben** (`Rational` in `total`/`mean`/`subtotal` und /act). Ursprünglich: ([act.rb:11](../lib/rdb/cmd/act.rb), `to_f` im /act-Teil
und in `total`). Abhilfe: `BigDecimal` oder ganzzahlige Rappen. und in `total`). Abhilfe: `BigDecimal` oder ganzzahlige Rappen.
4. **awk-Brücke** ([env.rb:119-138](../lib/rdb/env.rb)): ganze Tabelle als String 4. **awk-Brücke — bewusst offen.** `awk` und `awk_text` teilen sich jetzt den Aufruf. Streaming
und ein eigener Ausdrucksparser wären ein Umbau mit Verhaltensrisiko (awk-Syntax ist
dokumentierte Schnittstelle). Ursprünglich: ([env.rb:119-138](../lib/rdb/env.rb)): ganze Tabelle als String
durch `capture3`, kein Streaming; handgeschriebener Parser erkennt Strings und durch `capture3`, kein Streaming; handgeschriebener Parser erkennt Strings und
Regexe heuristisch (`prev`) und hängt vom awk-Dialekt ab. Regexe heuristisch (`prev`) und hängt vom awk-Dialekt ab.
5. **Globale Methoden auf `Object`** (`fast_opts`, `lock_file`, `num`, `money`, 5. **Globale Methoden auf `Object` — Behoben** (`Rdb::Helpers`, Konstanten in `Rdb`). Ursprünglich: (`fast_opts`, `lock_file`, `num`, `money`,
`ask`, `need`, `show_record` …). In ein Modul kapseln und in `Env` einbinden. `ask`, `need`, `show_record` …). In ein Modul kapseln und in `Env` einbinden.
6. **Doppelter Code.** `awk`/`awk_text`; Schlüsseltabellen-Parsing in `fast_opts`/ 6. **Doppelter Code — Behoben** (`Table.head_text`, `dash_cells`, `headed?`; `awk` über `awk_text`). Ursprünglich: `awk`/`awk_text`; Schlüsseltabellen-Parsing in `fast_opts`/
`key_columns`; von Hand gebaute Kopf-/Strichzeilen trotz `Table.dash_row`. `key_columns`; von Hand gebaute Kopf-/Strichzeilen trotz `Table.dash_row`.
7. **Tests.** Laufzeit ca. 66 s wegen eines Ruby-Prozesses pro Test. Fehlerfälle 7. **Tests — Behoben** (Suite 70 s → 21 s). Parallel, und `bin/rdb` startet mit `--disable-gems` (ca. 85 statt
170 ms je Befehl). Fehlerfälle haben jetzt Tests. Den Rest der Zeit braucht
`test_testall_runs_demo_script` (rund 300 Befehle nacheinander). Ursprünglich: Laufzeit ca. 66 s wegen eines Ruby-Prozesses pro Test. Fehlerfälle
(Encoding, Tab in Wert, Locks, Editor) nicht abgedeckt; Tests direkt gegen (Encoding, Tab in Wert, Locks, Editor) nicht abgedeckt; Tests direkt gegen
`Rdb.run` im Prozess wären schneller. `Rdb.run` im Prozess wären schneller.
8. **Repo.** `demo/fact.pl`, `fact_lc.pl`, `rule.pl` sind ungetrackt; 8. **Repo — offen, Entscheidung beim Autor.** Ursprünglich: `demo/fact.pl`, `fact_lc.pl`, `rule.pl` sind ungetrackt;
`demo/testall.old` (1187 Zeilen) wäre als Golden-File-Test besser gepflegt. `demo/testall.old` (1187 Zeilen) wäre als Golden-File-Test besser gepflegt.
## Empfohlene Reihenfolge ## Empfohlene Reihenfolge
Schwachstellen, Fehler und Verbesserung 1–3 sind erledigt. Offen sind 4–8: die Schwachstellen, Fehler und Verbesserungen 1–3 und 5–7 sind erledigt.
awk-Brücke, globale Methoden auf `Object`, doppelter Code, Testlaufzeit, Repo-Aufräumen. Offen: die awk-Brücke (4, bewusst) und das Repo-Aufräumen (8).
+19 -1
View File
@@ -3,7 +3,6 @@
# A Ruby re-creation of /rdb, the relational database tools Mark Williams # A Ruby re-creation of /rdb, the relational database tools Mark Williams
# Company sold for COHERENT (~1991). # Company sold for COHERENT (~1991).
require 'bigdecimal'
require 'open3' require 'open3'
require 'date' require 'date'
require 'tmpdir' require 'tmpdir'
@@ -33,6 +32,25 @@ module Rdb
hits.first hits.first
end end
# Exact decimal arithmetic with Rational (core Ruby, so bin/rdb can start
# without RubyGems): 0.1 + 0.2 is 0.3 and money adds up to the cent.
def decimal(str)
s = str.to_s.strip
return 0r if s.empty?
Rational(s.match?(NUM) ? s.sub(/\.\z/, '') : s.to_f.to_s)
end
# num with dec places, rounded half up ("%.2f" rounds the binary value).
def fixed(num, dec)
r = num.is_a?(Float) ? Rational(num.to_s) : num.to_r
r = r.round(dec, half: :up)
return r.to_i.to_s if dec.zero?
digits = (r.abs * 10**dec).to_i.to_s.rjust(dec + 1, '0')
"#{'-' if r.negative?}#{digits[0...-dec]}.#{digits[-dec..]}"
end
def die(msg) def die(msg)
abort "#{File.basename($PROGRAM_NAME)}: #{msg}" abort "#{File.basename($PROGRAM_NAME)}: #{msg}"
end end
+120 -101
View File
@@ -6,54 +6,57 @@
require 'stringio' require 'stringio'
require_relative '../fast' require_relative '../fast'
# Parse the common fast-access options. Stops at the first non-option, which module Rdb
# is left in args; returns the option hash. module Helpers
def fast_opts(args) # Parse the common fast-access options. Stops at the first non-option, which
o = { m: 's', h: false, l: false, s: false, x: false, exact: false, a: false, v: false, o: nil } # is left in args; returns the option hash.
until args.empty? def fast_opts(args)
a = args.first o = { m: 's', h: false, l: false, s: false, x: false, exact: false, a: false, v: false, o: nil }
case a until args.empty?
when /\A-m([bhirs]?)\z/ then o[:m] = Regexp.last_match(1).empty? ? 's' : Regexp.last_match(1) a = args.first
when '+x' then o[:exact] = true case a
when '-o' then args.shift; o[:o] = args.first when /\A-m([bhirs]?)\z/ then o[:m] = Regexp.last_match(1).empty? ? 's' : Regexp.last_match(1)
when /\A-[hslxav]+\z/ then a[1..].each_char { |c| o[c.to_sym] = true } when '+x' then o[:exact] = true
else break when '-o' then args.shift; o[:o] = args.first
when /\A-[hslxav]+\z/ then a[1..].each_char { |c| o[c.to_sym] = true }
else break
end
args.shift
end
o
end end
args.shift
end
o
end
def need_index(table, method) def need_index(table, method)
suffix = Rdb::Fast::SUFFIX[method] suffix = Rdb::Fast::SUFFIX[method]
return unless suffix && !File.exist?(table + suffix) return unless suffix && !File.exist?(table + suffix)
Rdb.die("no index #{table}#{suffix}: run index -m#{method} #{table}") Rdb.die("no index #{table}#{suffix}: run index -m#{method} #{table}")
end end
# The key columns and, unless keys come from the terminal, the keys. Without key # The key columns and, unless keys come from the terminal, the keys. Without key
# columns on the command line a key table names them in its head line # columns on the command line a key table names them in its head line
# (handbook p. 105: "search -mh mailtable < name"); otherwise the first column. # (handbook p. 105: "search -mh mailtable < name"); otherwise the first column.
def key_columns(args, data) def key_columns(args, data)
return [args.empty? ? [data.head[0]] : args, nil] if $stdin.tty? return [args.empty? ? [data.head[0]] : args, nil] if $stdin.tty?
text = $stdin.read.to_s text = $stdin.read.to_s
lines = text.lines.map { |l| l.chomp("\n") } lines = text.lines.map { |l| l.chomp("\n") }
cols = args cols = args
if cols.empty? if cols.empty?
keyed = lines.size >= 2 && Rdb::Table.dash_line?(lines[1].split("\t", -1)) cols = Rdb::Table.headed?(lines) ? lines[0].split("\t", -1).map(&:strip) : [data.head[0]]
cols = keyed ? lines[0].split("\t", -1).map(&:strip) : [data.head[0]] end
end [cols, Rdb::Fast.read_keys(StringIO.new(text), cols.size)]
[cols, Rdb::Fast.read_keys(StringIO.new(text), cols.size)] end
end
def each_key(ncols, keys = nil, &block) def each_key(ncols, keys = nil, &block)
if keys if keys
keys.each(&block) keys.each(&block)
elsif $stdin.tty? elsif $stdin.tty?
$stdin.each_line { |l| yield l.chomp.split("\t", -1).first(ncols) } $stdin.each_line { |l| yield l.chomp.split("\t", -1).first(ncols) }
else else
Rdb::Fast.read_keys($stdin, ncols).each { |k| yield k } Rdb::Fast.read_keys($stdin, ncols).each { |k| yield k }
end
end
end end
end end
@@ -76,7 +79,7 @@ Rdb.register('search', 'Search a table',
data = Rdb::Fast::Data.new(table) data = Rdb::Fast::Data.new(table)
need_index(table, o[:m]) need_index(table, o[:m])
keycols, keys = key_columns(args, data) keycols, keys = key_columns(args, data)
puts data.head.join("\t"), Rdb::Table.dash_row(data.head) unless o[:h] $stdout.write Rdb::Table.head_text(data.head) unless o[:h]
each_key(keycols.size, keys) do |key| each_key(keycols.size, keys) do |key|
hits = Rdb::Fast.search(data, o[:m], keycols, key, first: o[:s] || o[:l], partial: o[:x], exact: o[:exact]) hits = Rdb::Fast.search(data, o[:m], keycols, key, first: o[:s] || o[:l], partial: o[:x], exact: o[:exact])
hits.each do |h| hits.each do |h|
@@ -97,7 +100,7 @@ Rdb.register('seek', 'Return the beginning and ending offset of a row', 'seek [-
Rdb::Fast.search(data, o[:m], keycols, key, first: true).each do |h| Rdb::Fast.search(data, o[:m], keycols, key, first: true).each do |h|
puts "#{h.row.from} #{h.row.to} #{h.xfrom} #{h.xto}" puts "#{h.row.from} #{h.row.to} #{h.xfrom} #{h.xto}"
if o[:o] if o[:o]
File.write(o[:o], data.head.join("\t") + "\n" + Rdb::Table.dash_row(data.head) + "\n" + h.row.line + "\n") File.write(o[:o], Rdb::Table.head_text(data.head) + h.row.line + "\n")
end end
end end
end end
@@ -137,7 +140,7 @@ Rdb.register('replace', 'Insert a record into a file at specified location',
text = $stdin.read text = $stdin.read
lines = text.lines.map { |l| l.chomp("\n") } lines = text.lines.map { |l| l.chomp("\n") }
lines = lines.drop(2) if lines.size >= 2 && Rdb::Table.dash_line?(lines[1].split("\t", -1)) lines = lines.drop(2) if Rdb::Table.headed?(lines)
die 'replace: no input row' if lines.empty? die 'replace: no input row' if lines.empty?
keycol = Rdb::Fast::Data.new(table).head[0] keycol = Rdb::Fast::Data.new(table).head[0]
@@ -189,33 +192,41 @@ require 'shellwords'
# ---- locks ------------------------------------------------------------------- # ---- locks -------------------------------------------------------------------
# Lock files live in a private per-user directory, not loose in the shared module Rdb
# /tmp, so other users can neither forge nor delete them. module Helpers
def lock_dir # Lock files live in a private per-user directory, not loose in the shared
dir = File.join(Dir.tmpdir, "rdb-locks-#{Process.uid}") # /tmp, so other users can neither forge nor delete them.
Dir.mkdir(dir, 0o700) unless File.directory?(dir) def lock_dir
st = File.lstat(dir) dir = File.join(Dir.tmpdir, "rdb-locks-#{Process.uid}")
Rdb.die("#{dir}: not a private directory of yours") if st.symlink? || st.uid != Process.uid || (st.mode & 0o077) != 0 begin
dir Dir.mkdir(dir, 0o700)
end rescue Errno::EEXIST
nil # made by someone else, maybe at this very moment; checked below
end
st = File.lstat(dir)
Rdb.die("#{dir}: not a private directory of yours") if st.symlink? || st.uid != Process.uid || (st.mode & 0o077) != 0
dir
end
def lock_file(name) def lock_file(name)
File.join(lock_dir, "L#{File.basename(name)}.#{Digest::MD5.hexdigest(File.expand_path(name))[0, 8]}") File.join(lock_dir, "L#{File.basename(name)}.#{Digest::MD5.hexdigest(File.expand_path(name))[0, 8]}")
end end
# Read-modify-write of a lock file under an exclusive flock, so two lock or # Read-modify-write of a lock file under an exclusive flock, so two lock or
# unlock calls cannot interleave. The block gets the held locks and returns # unlock calls cannot interleave. The block gets the held locks and returns
# the new list of lines, or nil to leave the file alone. # the new list of lines, or nil to leave the file alone.
def with_lock_file(name) def with_lock_file(name)
File.open(lock_file(name), File::RDWR | File::CREAT, 0o600) do |f| File.open(lock_file(name), File::RDWR | File::CREAT, 0o600) do |f|
f.flock(File::LOCK_EX) f.flock(File::LOCK_EX)
held = f.read.lines(chomp: true).map { |l| l.split.map(&:to_i) } held = f.read.lines(chomp: true).map { |l| l.split.map(&:to_i) }
lines = yield held lines = yield held
next unless lines next unless lines
f.rewind f.rewind
f.truncate(0) f.truncate(0)
f.write(lines.map { |l| "#{l}\n" }.join) f.write(lines.map { |l| "#{l}\n" }.join)
end
end
end end
end end
@@ -267,20 +278,24 @@ end
# ---- interactive editing -------------------------------------------------------- # ---- interactive editing --------------------------------------------------------
def show_record(data_head, cells) module Rdb
data_head.each_with_index { |h, i| puts "#{i + 1} #{h}\t#{cells[i].to_s.strip}" } module Helpers
end def show_record(data_head, cells)
data_head.each_with_index { |h, i| puts "#{i + 1} #{h}\t#{cells[i].to_s.strip}" }
end
def edit_cells(head, cells) def edit_cells(head, cells)
require 'tempfile' require 'tempfile'
Tempfile.create('rdb-edit') do |f| Tempfile.create('rdb-edit') do |f|
f.write Rdb::Table.new(head, [cells], list: true).to_s f.write Rdb::Table.new(head, [cells], list: true).to_s
f.flush f.flush
system(*Shellwords.split(ENV['EDITOR'] || 'vi'), f.path) or Rdb.die("editor #{ENV['EDITOR'] || 'vi'} failed") system(*Shellwords.split(ENV['EDITOR'] || 'vi'), f.path) or Rdb.die("editor #{ENV['EDITOR'] || 'vi'} failed")
t = Rdb::Table.parse(File.read(f.path)) t = Rdb::Table.parse(File.read(f.path))
return nil if t.rows.empty? return nil if t.rows.empty?
head.map { |h| (j = t.head.index(h)) ? t.rows[0][j] : '' } head.map { |h| (j = t.head.index(h)) ? t.rows[0][j] : '' }
end
end
end end
end end
@@ -443,7 +458,7 @@ end
Rdb.register('validate', 'Find invalid data', "validate 'pattern { action } ...' < tableorlist") do |args| Rdb.register('validate', 'Find invalid data', "validate 'pattern { action } ...' < tableorlist") do |args|
die "usage: validate 'pattern { action } ...' < table" unless args.size == 1 die "usage: validate 'pattern { action } ...' < table" unless args.size == 1
t = table_in t = table_in
puts t.head.join("\t"), Rdb::Table.dash_row(t.head) $stdout.write Rdb::Table.head_text(t.head)
prog = "BEGIN{OFS=\"\\t\"} #{to_awk(args[0], t.head)} {print}" prog = "BEGIN{OFS=\"\\t\"} #{to_awk(args[0], t.head)} {print}"
out, ok = awk_text(prog, t.rows) out, ok = awk_text(prog, t.rows)
$stdout.write out $stdout.write out
@@ -512,27 +527,31 @@ end
# ---- exercising the fast-access methods --------------------------------------------- # ---- exercising the fast-access methods ---------------------------------------------
def sample_table(path, n = 2000) module Rdb
rows = (1..n).map { |i| [i.to_s, "name#{i * 7919 % n}", "city#{i % 97}"] } module Helpers
File.write(path, Rdb::Table.new(%w[Id Name City], rows).to_s) def sample_table(path, n = 2000)
end rows = (1..n).map { |i| [i.to_s, "name#{i * 7919 % n}", "city#{i % 97}"] }
File.write(path, Rdb::Table.new(%w[Id Name City], rows).to_s)
end
def time_methods(n = 2000) def time_methods(n = 2000)
Dir.mktmpdir do |dir| Dir.mktmpdir do |dir|
path = File.join(dir, 'unixtable') path = File.join(dir, 'unixtable')
sample_table(path, n) sample_table(path, n)
%w[s b h i r].each do |m| %w[s b h i r].each do |m|
t0 = Process.clock_gettime(Process::CLOCK_MONOTONIC) t0 = Process.clock_gettime(Process::CLOCK_MONOTONIC)
Rdb::Fast.build(path, m, 'Name') unless m == 's' Rdb::Fast.build(path, m, 'Name') unless m == 's'
t1 = Process.clock_gettime(Process::CLOCK_MONOTONIC) t1 = Process.clock_gettime(Process::CLOCK_MONOTONIC)
data = Rdb::Fast::Data.new(path) data = Rdb::Fast::Data.new(path)
hits = 0 hits = 0
(1..200).each do |i| (1..200).each do |i|
key = m == 'r' ? i.to_s : "name#{i * 13 % n}" key = m == 'r' ? i.to_s : "name#{i * 13 % n}"
hits += Rdb::Fast.search(data, m, ['Name'], key).size hits += Rdb::Fast.search(data, m, ['Name'], key).size
end
t2 = Process.clock_gettime(Process::CLOCK_MONOTONIC)
yield m, t1 - t0, t2 - t1, hits
end
end end
t2 = Process.clock_gettime(Process::CLOCK_MONOTONIC)
yield m, t1 - t0, t2 - t1, hits
end end
end end
end end
+61 -54
View File
@@ -5,43 +5,42 @@
# an accounting package: each command reads and writes plain /rdb tables in # an accounting package: each command reads and writes plain /rdb tables in
# the current directory. # the current directory.
ACT_COMMANDS = %w[act adjust balance bom calcpay calculate cashflow chartdup close consolidate cstate explode Rdb::ACT_COMMANDS = %w[act adjust balance bom calcpay calculate cashflow chartdup close consolidate cstate explode
fillform foot getjournal invoice sale tax update.inv w2].freeze fillform foot getjournal invoice sale tax update.inv w2].freeze
# Amounts are decimal numbers, not floats: sums of cents stay exact. module Rdb
def num(str) module Helpers
s = str.to_s.strip # Amounts are exact decimals (Rational), not floats: sums of cents stay exact.
s.match?(Rdb::NUM) ? BigDecimal(s.sub(/\.\z/, '')) : BigDecimal(s.to_f.to_s) def num(str)
end Rdb.decimal(str)
end
# Two decimals, always (money drops them from whole amounts). # Two decimals, always (money drops them from whole amounts).
def fixed2(val) def fixed2(val)
val = BigDecimal(val.to_s) unless val.is_a?(BigDecimal) Rdb.fixed(val, 2)
val.round(2, :half_up).to_s('F').then { |t| t.sub(/\.(\d*)\z/) { ".#{Regexp.last_match(1).ljust(2, '0')}" } } end
end
def money(val) def money(val)
val = BigDecimal(val.to_s) unless val.is_a?(BigDecimal) val == val.to_i ? val.to_i.to_s : Rdb.fixed(val, 2)
return val.to_i.to_s if val == val.to_i end
val.round(2, :half_up).to_s('F').sub(/\.(\d)\z/, '.\10') def need(file)
end File.exist?(file) || Rdb.die("#{file}: no such table (the /act commands work on tables in the current directory)")
Rdb::Table.open(file)
end
def need(file) def next_date
File.exist?(file) || Rdb.die("#{file}: no such table (the /act commands work on tables in the current directory)") ENV['NEXTDATE'] || Date.today.strftime('%y%m%d')
Rdb::Table.open(file) end
end
def next_date def account_key(acct)
ENV['NEXTDATE'] || Date.today.strftime('%y%m%d') [acct.to_f, acct]
end end
end
def account_key(acct)
[acct.to_f, acct]
end end
Rdb.register('act', 'List all /act commands', 'act') do |_args| Rdb.register('act', 'List all /act commands', 'act') do |_args|
names = ACT_COMMANDS.sort names = Rdb::ACT_COMMANDS.sort
w = names.map(&:size).max + 2 w = names.map(&:size).max + 2
names.each_slice(5) { |row| puts row.map { |n| n.ljust(w) }.join.rstrip } names.each_slice(5) { |row| puts row.map { |n| n.ljust(w) }.join.rstrip }
end end
@@ -66,7 +65,7 @@ Rdb.register('foot', 'Foot or subtotal Debits and Credit of ledger', 'foot') do
out = [] out = []
t.rows.sort_by { |r| account_key(r[0]) }.group_by { |r| r[0] }.each_value do |rows| t.rows.sort_by { |r| account_key(r[0]) }.group_by { |r| r[0] }.each_value do |rows|
out.concat(rows) out.concat(rows)
out << t.head.map { |h| '-' * [h.size, 1].max } out << Rdb::Table.dash_cells(t.head)
sub = [''] * t.head.size sub = [''] * t.head.size
sub[0] = rows.first[0] sub[0] = rows.first[0]
sub[d] = money(rows.sum { |r| num(r[d]) }) sub[d] = money(rows.sum { |r| num(r[d]) })
@@ -84,7 +83,7 @@ Rdb.register('adjust', 'Create adjusted trial balance table', 'adjust') do |_arg
m = (meta[h['Account']] ||= {}) m = (meta[h['Account']] ||= {})
h.each { |k, v| m[k] = v if !v.to_s.strip.empty? && !%w[Debit Credit].include?(k) && !m.key?(k) } h.each { |k, v| m[k] = v if !v.to_s.strip.empty? && !%w[Debit Credit].include?(k) && !m.key?(k) }
end end
net = Hash.new(BigDecimal(0)) net = Hash.new(0r)
[tb, ja].each do |t| [tb, ja].each do |t|
d = t.index_of('Debit') d = t.index_of('Debit')
c = t.index_of('Credit') c = t.index_of('Credit')
@@ -104,11 +103,15 @@ Rdb.register('adjust', 'Create adjusted trial balance table', 'adjust') do |_arg
File.write('adjustedtrial', Rdb::Table.new(head, rows).to_s) File.write('adjustedtrial', Rdb::Table.new(head, rows).to_s)
end end
def trial_parts(t) module Rdb
h = t.head module Helpers
fmt = h.index('Format') def trial_parts(t)
rows = t.rows.map { |r| { acct: r[0], debit: num(r[h.index('Debit')]), credit: num(r[h.index('Credit')]), fmt: r[fmt].to_s, name: r[h.index('Name')].to_s, row: r } } h = t.head
rows fmt = h.index('Format')
rows = t.rows.map { |r| { acct: r[0], debit: num(r[h.index('Debit')]), credit: num(r[h.index('Credit')]), fmt: r[fmt].to_s, name: r[h.index('Name')].to_s, row: r } }
rows
end
end
end end
Rdb.register('balance', 'Create balance sheet from adjusted trial balance', 'balance') do |_args| Rdb.register('balance', 'Create balance sheet from adjusted trial balance', 'balance') do |_args|
@@ -183,7 +186,7 @@ Rdb.register('fillform', 'Fill a tax from with adjusted trial balance data', 'fi
lines = need("taxform.#{args[0]}") lines = need("taxform.#{args[0]}")
trial = need('adjustedtrial') trial = need('adjustedtrial')
ti = trial.index_of('Taxline') ti = trial.index_of('Taxline')
sums = Hash.new(BigDecimal(0)) sums = Hash.new(0r)
trial.rows.each { |r| sums[r[ti]] += (num(r[trial.index_of('Debit')]) - num(r[trial.index_of('Credit')])).abs } trial.rows.each { |r| sums[r[ti]] += (num(r[trial.index_of('Debit')]) - num(r[trial.index_of('Credit')])).abs }
rows = lines.rows.map { |r| [r[0], money(sums[r[0]]), r[1].to_s] } rows = lines.rows.map { |r| [r[0], money(sums[r[0]]), r[1].to_s] }
File.write(args[0], Rdb::Table.new(%w[Taxline Amount Description], rows).to_s) File.write(args[0], Rdb::Table.new(%w[Taxline Amount Description], rows).to_s)
@@ -205,14 +208,14 @@ Rdb.register('tax', 'Compute tax from income and tax table', 'tax income < taxta
income = num(args[0]) income = num(args[0])
rows = table_in.rows.map { |r| r.map { |c| num(c) } }.sort_by(&:first) rows = table_in.rows.map { |r| r.map { |c| num(c) } }.sort_by(&:first)
row = rows.reverse.find { |r| r[0] <= income } row = rows.reverse.find { |r| r[0] <= income }
puts format('%7d', row ? (row[1] + (income - row[0]) * row[2] / 100.0).round : 0) puts format('%7d', row ? (row[1] + (income - row[0]) * row[2] / 100).round(half: :up) : 0)
end end
Rdb.register('cashflow', 'Compute balance column of cash table', 'cashflow < cashtable') do |_args| Rdb.register('cashflow', 'Compute balance column of cash table', 'cashflow < cashtable') do |_args|
t = table_in t = table_in
a = t.index_of('Amount') a = t.index_of('Amount')
b = t.index_of('Balance') b = t.index_of('Balance')
bal = BigDecimal(0) bal = 0r
rows = t.rows.map do |r| rows = t.rows.map do |r|
r = r.dup r = r.dup
if r[a].strip.empty? && !r[b].strip.empty? if r[a].strip.empty? && !r[b].strip.empty?
@@ -250,9 +253,9 @@ end
Rdb.register('bom', 'Produce bill-of-materials from parts list', 'bom') do |_args| Rdb.register('bom', 'Produce bill-of-materials from parts list', 'bom') do |_args|
part = need('part') part = need('part')
items = need('saleitem') items = need('saleitem')
qty = Hash.new(BigDecimal(0)) qty = Hash.new(0r)
items.rows.each { |r| qty[r[items.index_of('Code')]] += num(r[items.index_of('Qty')]) } items.rows.each { |r| qty[r[items.index_of('Code')]] += num(r[items.index_of('Qty')]) }
need_total = Hash.new(BigDecimal(0)) need_total = Hash.new(0r)
part.rows.each do |r| part.rows.each do |r|
code = r[part.index_of('Code')] code = r[part.index_of('Code')]
need_total[r[part.index_of('Subpart')]] += qty[code] * num(r[part.index_of('Count')]) if qty.key?(code) need_total[r[part.index_of('Subpart')]] += qty[code] * num(r[part.index_of('Count')]) if qty.key?(code)
@@ -263,7 +266,7 @@ end
Rdb.register('explode', 'Produce table of subparts and their count for a part', 'explode Part Amount [ table ]') do |args| Rdb.register('explode', 'Produce table of subparts and their count for a part', 'explode Part Amount [ table ]') do |args|
die 'usage: explode Part Amount [table]' if args.size < 2 die 'usage: explode Part Amount [table]' if args.size < 2
t = need(args[2] || 'parttable') t = need(args[2] || 'parttable')
totals = Hash.new(BigDecimal(0)) totals = Hash.new(0r)
queue = [[args[0], num(args[1])]] queue = [[args[0], num(args[1])]]
until queue.empty? until queue.empty?
part, amount = queue.shift part, amount = queue.shift
@@ -275,20 +278,24 @@ end
# ---- sales ------------------------------------------------------------------------------ # ---- sales ------------------------------------------------------------------------------
def ask(prompt) module Rdb
print prompt module Helpers
$stdout.flush def ask(prompt)
($stdin.gets || '').strip print prompt
end $stdout.flush
($stdin.gets || '').strip
end
def address_block(customers, number) def address_block(customers, number)
c = customers.rows.find { |r| r[0] == number } c = customers.rows.find { |r| r[0] == number }
return ["Customer Number #{number}"] unless c return ["Customer Number #{number}"] unless c
h = customers.head.zip(c).to_h h = customers.head.zip(c).to_h
["Mailing Address for Customer Number #{number}", ["Mailing Address for Customer Number #{number}",
[h['Title'], h['First'], h['Last'], h['Position']].compact.join(' ').strip, [h['Title'], h['First'], h['Last'], h['Position']].compact.join(' ').strip,
h['Company'], h['Street'], [h['City'], h['State'], h['Country']].compact.join(' ')].compact.reject(&:empty?) h['Company'], h['Street'], [h['City'], h['State'], h['Country']].compact.join(' ')].compact.reject(&:empty?)
end
end
end end
Rdb.register('invoice', 'Print invoice for a sale order', 'invoice') do |_args| Rdb.register('invoice', 'Print invoice for a sale order', 'invoice') do |_args|
@@ -359,7 +366,7 @@ Rdb.register('sale', 'Enter sale order, and item, update customer', 'sale') do |
end end
puts address_block(customers, cust) puts address_block(customers, cust)
n = Rdb::Table.open('salesorder').rows.map { |r| r[0].to_i }.max.to_i + 1 n = Rdb::Table.open('salesorder').rows.map { |r| r[0].to_i }.max.to_i + 1
gross = BigDecimal(0) gross = 0r
items = [] items = []
loop do loop do
code = ask('Code (Return when done): ') code = ask('Code (Return when done): ')
@@ -375,7 +382,7 @@ Rdb.register('sale', 'Enter sale order, and item, update customer', 'sale') do |
end end
next if items.empty? next if items.empty?
tax = gross * BigDecimal('0.05') tax = gross * Rational('0.05')
order = [n.to_s, cust, next_date, '0', fixed2(gross), fixed2(tax), fixed2(gross + tax)] order = [n.to_s, cust, next_date, '0', fixed2(gross), fixed2(tax), fixed2(gross + tax)]
puts Rdb::Table.new(%w[Number Cust Date Shipped Gross Tax Total], [order]).to_s puts Rdb::Table.new(%w[Number Cust Date Shipped Gross Tax Total], [order]).to_s
if ask('Is this correct? (y, n) ').downcase.start_with?('y') if ask('Is this correct? (y, n) ').downcase.start_with?('y')
+17 -13
View File
@@ -105,21 +105,25 @@ end
# ---- names that sound alike ---------------------------------------------------- # ---- names that sound alike ----------------------------------------------------
def soundex_code(word) module Rdb
codes = { 'BFPV' => '1', 'CGJKQSXZ' => '2', 'DT' => '3', 'L' => '4', 'MN' => '5', 'R' => '6' } module Helpers
letters = word.upcase.gsub(/[^A-Z]/, '').chars def soundex_code(word)
return '' if letters.empty? codes = { 'BFPV' => '1', 'CGJKQSXZ' => '2', 'DT' => '3', 'L' => '4', 'MN' => '5', 'R' => '6' }
letters = word.upcase.gsub(/[^A-Z]/, '').chars
return '' if letters.empty?
code = ->(c) { codes.find { |k, _| k.include?(c) }&.last } code = ->(c) { codes.find { |k, _| k.include?(c) }&.last }
out = letters[0].dup out = letters[0].dup
last = code.(letters[0]) last = code.(letters[0])
letters.drop(1).each do |c| letters.drop(1).each do |c|
d = code.(c) d = code.(c)
out << d if d && d != last out << d if d && d != last
last = d unless 'HW'.include?(c) last = d unless 'HW'.include?(c)
break if out.size == 4 break if out.size == 4
end
out.ljust(4, '0')
end
end end
out.ljust(4, '0')
end end
Rdb.register('soundex', 'Convert a name into soundex code', 'soundex < file > file.x ; index -mb file.x Soundex') do |_args| Rdb.register('soundex', 'Convert a name into soundex code', 'soundex < file > file.x ; index -mb file.x Soundex') do |_args|
+4 -4
View File
@@ -189,7 +189,7 @@ end
# total, mean, maximum, minimum, width, precision and datatype are one program # total, mean, maximum, minimum, width, precision and datatype are one program
# that decides what to compute from the name it was called by. # that decides what to compute from the name it was called by.
STATS = { Rdb::STATS = {
'total' => 'Sum a column', 'total' => 'Sum a column',
'mean' => 'Display the mean of a column', 'mean' => 'Display the mean of a column',
'maximum' => 'Display the maximum value in a column', 'maximum' => 'Display the maximum value in a column',
@@ -199,7 +199,7 @@ STATS = {
'datatype' => 'Display the data type of each column selected' 'datatype' => 'Display the data type of each column selected'
}.freeze }.freeze
STATS.each do |cmd, desc| Rdb::STATS.each do |cmd, desc|
Rdb.register(cmd, desc, "#{cmd} [ -l ] [ column ... ] < table") do |args| Rdb.register(cmd, desc, "#{cmd} [ -l ] [ column ... ] < table") do |args|
list = args.first == '-l' list = args.first == '-l'
args.shift if list args.shift if list
@@ -209,7 +209,7 @@ STATS.each do |cmd, desc|
sel.each do |i| sel.each do |i|
vals = t.rows.map { |r| r[i] } vals = t.rows.map { |r| r[i] }
filled = vals.reject { |v| v.strip.empty? } filled = vals.reject { |v| v.strip.empty? }
nums = filled.all? { |v| numeric?(v) } ? vals.map { |v| v.strip.empty? ? BigDecimal(0) : dec_num(v) } : nil nums = filled.all? { |v| numeric?(v) } ? vals.map { |v| dec_num(v) } : nil
dec = filled.map { |v| decimals(v) }.max.to_i dec = filled.map { |v| decimals(v) }.max.to_i
result[i] = result[i] =
case name case name
@@ -260,7 +260,7 @@ Rdb.register('subtotal', 'Output subtotals of columns in a table',
end end
if list if list
out.concat(group) out.concat(group)
out << t.head.map { |h| '-' * [h.size, 1].max } out << Rdb::Table.dash_cells(t.head)
end end
out << sub out << sub
group = [] group = []
+24 -16
View File
@@ -4,19 +4,23 @@
require 'shellwords' require 'shellwords'
# Fill in a form for one row: <Column> is replaced by the value, <!command!> module Rdb
# by the output of the shell command, and <!cmd <Column>!> passes data to it. module Helpers
# Values inside a command are shell-escaped, so a cell can never add commands # Fill in a form for one row: <Column> is replaced by the value, <!command!>
# of its own; they arrive as one word each (do not put them in quotes). # by the output of the shell command, and <!cmd <Column>!> passes data to it.
def fill_form(form, head, row) # Values inside a command are shell-escaped, so a cell can never add commands
values = head.zip(row).to_h { |h, v| [h.strip, v.to_s] } # of its own; they arrive as one word each (do not put them in quotes).
form.gsub(/<!(.*?)!>|<([^<>!\n]*)>/m) do def fill_form(form, head, row)
if Regexp.last_match(1) values = head.zip(row).to_h { |h, v| [h.strip, v.to_s] }
cmd = Regexp.last_match(1).gsub(/<([^<>!\n]*)>/) { Shellwords.escape(values.fetch(Regexp.last_match(1).strip, Regexp.last_match(0))) } form.gsub(/<!(.*?)!>|<([^<>!\n]*)>/m) do
`#{cmd}` if Regexp.last_match(1)
else cmd = Regexp.last_match(1).gsub(/<([^<>!\n]*)>/) { Shellwords.escape(values.fetch(Regexp.last_match(1).strip, Regexp.last_match(0))) }
name = Regexp.last_match(2).strip `#{cmd}`
values.key?(name) ? values[name] : Regexp.last_match(0) else
name = Regexp.last_match(2).strip
values.key?(name) ? values[name] : Regexp.last_match(0)
end
end
end end
end end
end end
@@ -76,9 +80,13 @@ Rdb.register('screen', 'Convert a form into a screen-input shell program', 'scre
puts ': append row', %(echo "#{names.map { |x| "$#{x}" }.join("\t")}") puts ': append row', %(echo "#{names.map { |x| "$#{x}" }.join("\t")}")
end end
def label_lines(head, row) module Rdb
h = head.zip(row).to_h module Helpers
[h['Name'], h['Company'], h['Street'], "#{h['City']}, #{h['State']} #{h['ZIP']}".strip].compact def label_lines(head, row)
h = head.zip(row).to_h
[h['Name'], h['Company'], h['Street'], "#{h['City']}, #{h['State']} #{h['ZIP']}".strip].compact
end
end
end end
Rdb.register('label', 'Print mailing labels from a mailing list', 'label < list') do |_args| Rdb.register('label', 'Print mailing labels from a mailing list', 'label < list') do |_args|
+68 -60
View File
@@ -4,20 +4,24 @@ require 'shellwords'
# Help, manual pages, terminal helpers, menus and housekeeping. # Help, manual pages, terminal helpers, menus and housekeeping.
ESC = "\e" Rdb::ESC = "\e"
TERM_CAPS = { Rdb::TERM_CAPS = {
'cl' => "#{ESC}[H#{ESC}[2J", 'ho' => "#{ESC}[H", 'ce' => "#{ESC}[K", 'cd' => "#{ESC}[J", 'cl' => "#{Rdb::ESC}[H#{Rdb::ESC}[2J", 'ho' => "#{Rdb::ESC}[H", 'ce' => "#{Rdb::ESC}[K", 'cd' => "#{Rdb::ESC}[J",
'cm' => "#{ESC}[%i%d;%dH", 'so' => "#{ESC}[7m", 'se' => "#{ESC}[m", 'us' => "#{ESC}[4m", 'cm' => "#{Rdb::ESC}[%i%d;%dH", 'so' => "#{Rdb::ESC}[7m", 'se' => "#{Rdb::ESC}[m", 'us' => "#{Rdb::ESC}[4m",
'ue' => "#{ESC}[m", 'mb' => "#{ESC}[5m", 'md' => "#{ESC}[1m", 'me' => "#{ESC}[m", 'ue' => "#{Rdb::ESC}[m", 'mb' => "#{Rdb::ESC}[5m", 'md' => "#{Rdb::ESC}[1m", 'me' => "#{Rdb::ESC}[m",
'as' => "#{ESC}(0", 'ae' => "#{ESC}(B", 'up' => "#{ESC}[A", 'do' => "#{ESC}[B", 'bs' => "\b" 'as' => "#{Rdb::ESC}(0", 'ae' => "#{Rdb::ESC}(B", 'up' => "#{Rdb::ESC}[A", 'do' => "#{Rdb::ESC}[B", 'bs' => "\b"
}.freeze }.freeze
def command_rows module Rdb
Rdb::COMMANDS.values.reject { |c| c.name.include?('.list') || c.name == 'rdb.install' }.sort_by(&:name) module Helpers
end def command_rows
Rdb::COMMANDS.values.reject { |c| c.name.include?('.list') || c.name == 'rdb.install' }.sort_by(&:name)
end
def syntax_of(cmd) def syntax_of(cmd)
cmd.syntax.sub(/\A\S+\s*/, '') cmd.syntax.sub(/\A\S+\s*/, '')
end
end
end end
Rdb.register('rdb.list', 'List all /rdb programs', 'rdb') do |_args| Rdb.register('rdb.list', 'List all /rdb programs', 'rdb') do |_args|
@@ -80,56 +84,60 @@ end
# ---- manual pages --------------------------------------------------------------- # ---- manual pages ---------------------------------------------------------------
def troff_clean(str) module Rdb
# a literal backslash (\\ or \e) is parked in \0 so that later rules cannot see it module Helpers
str.gsub('\\\\', "\0").gsub(/\\f[BIRP]/, '').gsub('\\e', "\0").gsub('\\(em', ' -- ').gsub('\\(aa', "'") def troff_clean(str)
.gsub('\\-', '-').gsub('\\&', '').gsub('\\*(UX', 'UNIX').gsub('``', '"').gsub("''", '"').gsub('\\ ', ' ') # a literal backslash (\\ or \e) is parked in \0 so that later rules cannot see it
.gsub(/\\\(..|\\\*\(..|\\[a-z]/, '').gsub("\0", '\\') str.gsub('\\\\', "\0").gsub(/\\f[BIRP]/, '').gsub('\\e', "\0").gsub('\\(em', ' -- ').gsub('\\(aa', "'")
end .gsub('\\-', '-').gsub('\\&', '').gsub('\\*(UX', 'UNIX').gsub('``', '"').gsub("''", '"').gsub('\\ ', ' ')
.gsub(/\\\(..|\\\*\(..|\\[a-z]/, '').gsub("\0", '\\')
end
def render_man(src) def render_man(src)
out = [] out = []
para = [] para = []
verbatim = false verbatim = false
flush = lambda do flush = lambda do
unless para.empty? unless para.empty?
words = para.join(' ').split words = para.join(' ').split
line = '' line = ''
words.each { |w| line.size + w.size + 1 > 72 ? (out << line; line = w) : line = (line.empty? ? w : "#{line} #{w}") } words.each { |w| line.size + w.size + 1 > 72 ? (out << line; line = w) : line = (line.empty? ? w : "#{line} #{w}") }
out << line unless line.empty? out << line unless line.empty?
para.clear para.clear
end end
end end
src.each_line do |raw| src.each_line do |raw|
l = raw.chomp l = raw.chomp
if l.start_with?('.') if l.start_with?('.')
macro, rest = l[1..].split(/\s+/, 2) macro, rest = l[1..].split(/\s+/, 2)
rest = rest.to_s.strip.delete_prefix('"').delete_suffix('"') rest = rest.to_s.strip.delete_prefix('"').delete_suffix('"')
case macro case macro
when 'TH', 'II', 'ta', 'ne', 'R', 'RE', 'SO', 'B1', 'B2', 'nf', 'fi', 'br' then nil when 'TH', 'II', 'ta', 'ne', 'R', 'RE', 'SO', 'B1', 'B2', 'nf', 'fi', 'br' then nil
when '\\"' then nil when '\\"' then nil
when 'PC' then flush.(); out << '' when 'PC' then flush.(); out << ''
when 'SH' then flush.(); out << '' << troff_clean(rest).upcase when 'SH' then flush.(); out << '' << troff_clean(rest).upcase
when 'PP', 'sp' then flush.(); out << '' when 'PP', 'sp' then flush.(); out << ''
when 'DM', 'DS' then flush.(); verbatim = true when 'DM', 'DS' then flush.(); verbatim = true
when 'DE' then verbatim = false when 'DE' then verbatim = false
when 'IP' then flush.(); out << ''; para << troff_clean(rest.split(/\s+\d/).first.to_s) when 'IP' then flush.(); out << ''; para << troff_clean(rest.split(/\s+\d/).first.to_s)
when 'B', 'I', 'BR', 'IR', 'RB', 'RI' then para << troff_clean(rest.delete('"')).gsub(/\s+/, ' ') unless verbatim when 'B', 'I', 'BR', 'IR', 'RB', 'RI' then para << troff_clean(rest.delete('"')).gsub(/\s+/, ' ') unless verbatim
else nil else nil
end
elsif verbatim
out << " #{troff_clean(l)}"
elsif l.start_with?("\t", ' ')
flush.()
out << " #{troff_clean(l.strip)}"
elsif l.strip.empty?
flush.()
else
para << troff_clean(l.strip)
end
end end
elsif verbatim
out << " #{troff_clean(l)}"
elsif l.start_with?("\t", ' ')
flush.() flush.()
out << " #{troff_clean(l.strip)}" out.join("\n").squeeze("\n")
elsif l.strip.empty?
flush.()
else
para << troff_clean(l.strip)
end end
end end
flush.()
out.join("\n").squeeze("\n")
end end
Rdb.register('man', 'Display the manual page for a command', 'man [-g] command') do |args| Rdb.register('man', 'Display the manual page for a command', 'man [-g] command') do |args|
@@ -161,21 +169,21 @@ end
Rdb.register('termput', 'Get terminal capability from /etc/termcap file', 'termput capability') do |args| Rdb.register('termput', 'Get terminal capability from /etc/termcap file', 'termput capability') do |args|
die 'usage: termput capability' unless args.size == 1 die 'usage: termput capability' unless args.size == 1
$stdout.print TERM_CAPS.fetch(args[0]) { die "no such capability '#{args[0]}'" } $stdout.print Rdb::TERM_CAPS.fetch(args[0]) { die "no such capability '#{args[0]}'" }
end end
Rdb.register('tset', 'Fetch termcap entry for a terminal type', 'tset [ TERM ]') do |args| Rdb.register('tset', 'Fetch termcap entry for a terminal type', 'tset [ TERM ]') do |args|
term = args.first || ENV['TERM'] || 'ansi' term = args.first || ENV['TERM'] || 'ansi'
puts ":#{TERM_CAPS.map { |k, v| "#{k}=#{v.inspect[1..-2]}" }.join(':')}:co##{ENV['COLUMNS'] || 80}:li##{ENV['LINES'] || 24}:tc=#{term}:" puts ":#{Rdb::TERM_CAPS.map { |k, v| "#{k}=#{v.inspect[1..-2]}" }.join(':')}:co##{ENV['COLUMNS'] || 80}:li##{ENV['LINES'] || 24}:tc=#{term}:"
end end
Rdb.register('clear.rdb', "Clear the terminal's screen", 'clear.rdb') do |_args| Rdb.register('clear.rdb', "Clear the terminal's screen", 'clear.rdb') do |_args|
$stdout.print TERM_CAPS['cl'] $stdout.print Rdb::TERM_CAPS['cl']
end end
Rdb.register('cursor', 'Move the cursor to the row and column requested', 'cursor row col [ CURSOR ]') do |args| Rdb.register('cursor', 'Move the cursor to the row and column requested', 'cursor row col [ CURSOR ]') do |args|
die 'usage: cursor row col [CURSOR]' if args.size < 2 die 'usage: cursor row col [CURSOR]' if args.size < 2
$stdout.print "#{ESC}[#{args[0].to_i};#{args[1].to_i}H" $stdout.print "#{Rdb::ESC}[#{args[0].to_i};#{args[1].to_i}H"
$stdout.flush $stdout.flush
end end
+14 -10
View File
@@ -2,16 +2,20 @@
# Formatting tables and working with text. # Formatting tables and working with text.
def justify_numbers(cells) module Rdb
parts = cells.map { |c| c.empty? ? ['', nil] : c.split('.', 2) } module Helpers
int_w = parts.map { |i, _| i.size }.max def justify_numbers(cells)
frac_w = parts.map { |_, f| f ? f.size : 0 }.max parts = cells.map { |c| c.empty? ? ['', nil] : c.split('.', 2) }
cells.zip(parts).map do |c, (i, f)| int_w = parts.map { |i, _| i.size }.max
next ' ' * (int_w + (frac_w.zero? ? 0 : frac_w + 1)) if c.empty? frac_w = parts.map { |_, f| f ? f.size : 0 }.max
cells.zip(parts).map do |c, (i, f)|
next ' ' * (int_w + (frac_w.zero? ? 0 : frac_w + 1)) if c.empty?
s = i.rjust(int_w) s = i.rjust(int_w)
s += f ? ".#{f.ljust(frac_w)}" : ' ' * (frac_w.zero? ? 0 : frac_w + 1) s += f ? ".#{f.ljust(frac_w)}" : ' ' * (frac_w.zero? ? 0 : frac_w + 1)
s s
end
end
end end
end end
@@ -112,7 +116,7 @@ Rdb.register('pad', 'Add extra spaces at end of last column', 'pad [-number] < t
if t.list if t.list
t.with(rows: t.rows.map { |r| r[0...-1] + [r.last + ' ' * n] }).write t.with(rows: t.rows.map { |r| r[0...-1] + [r.last + ' ' * n] }).write
else else
$stdout.write t.head.join("\t"), "\n", Rdb::Table.dash_row(t.head), "\n" $stdout.write Rdb::Table.head_text(t.head)
t.rows.each { |r| puts (r[0...-1] + [r.last + ' ' * n]).join("\t") } t.rows.each { |r| puts (r[0...-1] + [r.last + ' ' * n]).join("\t") }
end end
end end
+10 -11
View File
@@ -1,8 +1,13 @@
# frozen_string_literal: true # frozen_string_literal: true
module Rdb module Rdb
# Helpers of single command files (lib/rdb/cmd/*.rb) go in here, not on
# Object; every Env includes them.
module Helpers; end
# Command blocks run inside an Env, which holds the helpers they share. # Command blocks run inside an Env, which holds the helpers they share.
class Env class Env
include Helpers
attr_reader :name attr_reader :name
def initialize(name) def initialize(name)
@@ -30,14 +35,12 @@ module Rdb
str.strip[/\.(\d+)\z/, 1].to_s.size str.strip[/\.(\d+)\z/, 1].to_s.size
end end
# Decimal arithmetic, so that 0.1 + 0.2 is 0.3 and money adds up to the cent.
def dec_num(str) def dec_num(str)
BigDecimal(str.strip.sub(/\.\z/, '')) Rdb.decimal(str)
end end
def fmt(num, dec) def fmt(num, dec)
num = BigDecimal(num.to_s) unless num.is_a?(BigDecimal) Rdb.fixed(num, dec)
dec.zero? ? num.round(0, :half_up).to_i.to_s : num.round(dec, :half_up).to_s('F').then { |t| t.sub(/\.(\d*)\z/) { ".#{Regexp.last_match(1).ljust(dec, '0')}" } }
end end
def blank_row?(row) def blank_row?(row)
@@ -102,18 +105,14 @@ module Rdb
# Run an awk program over rows (tab separated) and return the output rows. # Run an awk program over rows (tab separated) and return the output rows.
def awk(prog, rows) def awk(prog, rows)
input = rows.map { |r| r.join("\t") }.join("\n") out, ok = awk_text(prog, rows)
input += "\n" unless input.empty? exit 1 unless ok
out, err, st = Open3.capture3('awk', '-F', "\t", prog, stdin_data: input)
$stderr.print err unless err.empty?
exit 1 unless st.success?
out.each_line.map { |l| l.chomp.split("\t", -1) } out.each_line.map { |l| l.chomp.split("\t", -1) }
end end
# Raw output of awk, for programs that print their own text. # Raw output of awk, for programs that print their own text.
def awk_text(prog, rows) def awk_text(prog, rows)
input = rows.map { |r| r.join("\t") }.join("\n") input = rows.map { |r| "#{r.join("\t")}\n" }.join
input += "\n" unless input.empty?
out, err, st = Open3.capture3('awk', '-F', "\t", prog, stdin_data: input) out, err, st = Open3.capture3('awk', '-F', "\t", prog, stdin_data: input)
$stderr.print err unless err.empty? $stderr.print err unless err.empty?
[out, st.success?] [out, st.success?]
+2 -2
View File
@@ -129,7 +129,7 @@ module Rdb
when 'b' when 'b'
k = data.column(keycol) k = data.column(keycol)
body = live_rows(data).sort_by { |r| r.cells[k].to_s.strip }.map { |r| r.line + "\n" } body = live_rows(data).sort_by { |r| r.cells[k].to_s.strip }.map { |r| r.line + "\n" }
Rdb.write_file(path, data.head.join("\t") + "\n" + Table.dash_row(data.head) + "\n" + body.join) Rdb.write_file(path, Table.head_text(data.head) + body.join)
end end
end end
@@ -393,7 +393,7 @@ module Rdb
return [] if text.empty? return [] if text.empty?
lines = text.lines.map { |l| l.chomp("\n") } lines = text.lines.map { |l| l.chomp("\n") }
if lines.size >= 2 && Table.dash_line?(lines[1].split("\t", -1)) if Table.headed?(lines)
lines.drop(2).reject(&:empty?).map { |l| l.split("\t", -1).first(ncols) } lines.drop(2).reject(&:empty?).map { |l| l.split("\t", -1).first(ncols) }
else else
lines.reject(&:empty?).map { |l| l.split("\t", -1).first(ncols) } lines.reject(&:empty?).map { |l| l.split("\t", -1).first(ncols) }
+16 -4
View File
@@ -28,6 +28,11 @@ module Rdb
end end
end end
# Do these lines (chomped) start with a head line and a dash line?
def self.headed?(lines)
lines.size >= 2 && dash_line?(lines[1].split("\t", -1))
end
def self.parse(text) def self.parse(text)
text = utf8(text) text = utf8(text)
return parse_list(text) if text.start_with?("\n") return parse_list(text) if text.start_with?("\n")
@@ -78,15 +83,22 @@ module Rdb
def to_s def to_s
return list_text if @list return list_text if @list
out = +'' out = Table.head_text(@head)
out << @head.join("\t") << "\n"
out << Table.dash_row(@head) << "\n"
@rows.each { |r| out << r.join("\t") << "\n" } @rows.each { |r| out << r.join("\t") << "\n" }
out out
end end
def self.dash_cells(head)
head.map { |h| '-' * [h.size, 1].max }
end
def self.dash_row(head) def self.dash_row(head)
head.map { |h| '-' * [h.size, 1].max }.join("\t") dash_cells(head).join("\t")
end
# Head line and dash line, each with its newline.
def self.head_text(head)
+"#{head.join("\t")}\n#{dash_row(head)}\n"
end end
def list_text def list_text
+4
View File
@@ -12,6 +12,10 @@ DEMO = File.join(ROOT, 'demo')
# Each test runs in a scratch copy of demo/, so commands that change files # Each test runs in a scratch copy of demo/, so commands that change files
# (append, delete, replace ...) never touch the originals. # (append, delete, replace ...) never touch the originals.
class RdbCase < Minitest::Test class RdbCase < Minitest::Test
# Tests spawn their own processes in their own directory, so they can run
# side by side (N=1 in MT_CPU runs them one at a time).
parallelize_me!
def setup def setup
@dir = Dir.mktmpdir('rdb-test') @dir = Dir.mktmpdir('rdb-test')
FileUtils.cp_r(Dir[File.join(DEMO, '*')], @dir) FileUtils.cp_r(Dir[File.join(DEMO, '*')], @dir)