Flay hook ignores flay crashes by only reading stdout

[sofmeireles]

The Flay pre-commit hook runs flay --mass <threshold> --fuzzy 1 <file> on each modified file. The problem is that flay can crash internally (there's a bug in flay 2.13.3 where --fuzzy causes a NoMethodError on certain Ruby AST structures), and when that happens the hook silently passes.

The crash goes to stderr and exits non-zero, but the plugin only checks result.stdout — which is empty when flay crashes — so it assumes nothing was found.

You can reproduce it with a file containing two similar methods with .map(&:to_s) or similar block-passing patterns, like:

def first_filter_valid
  return if first.blank?
  return if first.all? { |v| v.in?(SOME_CONST.keys.map(&:to_s)) }
  errors.add(:first, :invalid)
end

def second_filter_valid
  return if second.blank?
  return if second.all? { |v| v.in?(SOME_CONST.keys.map(&:to_s)) }
  errors.add(:second, :invalid)
end

Enable the Flay hook and run overcommit --run — it'll pass even though flay crashes with:

/home/.../flay-2.13.3/lib/flay.rb:318:in 'Flay#process_fuzzy':
  undefined method 'modified=' for an instance of Array (NoMethodError)
    tmpl.modified = true

The relevant code is at lib/overcommit/hook/pre_commit/flay.rb:24-28:

result = execute(command, args: [file])
results = result.stdout.split("\n\n")
results.shift
unless results.empty?

result.status and result.stderr are never checked, so crashes are invisible. Would be great if the plugin could also look at those to catch cases where flay itself fails.