I have recently been developing
Spreadsheet::HTML and apart from a handful of mistakes, two particular CPANTS smoke tests (see
this and
that) always have one specific test fail. And only that one test. Here is the relevant output:
|
|
# Failed test 'encoding all HTML entities' # at t/03-encode-data.t line 42. # got: '<table><tr><th><</th><th>=</th><th>&</th><th>></th><th>"</th><th>'</th></tr><tr><td><</td><td>=</td><td>&</td><td>></td><td>"</td><td>'</td></tr></table>' # expected: '<table><tr><th><</th><th>=</th><th>&</th><th>></th><th>"</th><th>'</th></tr><tr><td><</td><td>=</td><td>&</td><td>></td><td>"</td><td>'</td></tr></table>' |
Needless to say, i was hard pressed to spot the problem. I usually just ignored these two servers and moved on, but i always kept the notion to eventually solve this issue in the back of my mind. I even once ran the two strings through
uniq and it too saw no difference. Finally, today i realized that i was looking at HTML, so i inspected the source and sure enough, there it was! It was very hard to spot, but a
' character was not being encoded into its ASCII HTML entities equivalent,
'. Now that the problem is actually visible, a solution can be worked towards. I suspected the problem was that an older version of HTML::Entities was installed on the two smoke test servers that trigger this failure, but Spreadsheet::HTML does not require HTML::Entities, instead it requires
HTML::AutoTag (another one of my modules) which delegates the work to HTML::Entities. So i immediately checked the Makefile for HTML::AutoTag and sure enough i was not specifying any version for its build. However! ... because Spreadsheet::HTML is not directly requiring HTML::Entities in its Makefile (as it should not), the test reports will not disclose anything about that module, so i now needed to determine what versions were being used.
The next step was completely manual, while a programmatic solution could likely be developed, it seemed easy enough to pop on over to the latest smoke test reports for HTML::AutoTag and just search the web page for the same Perl and OS versions for the two failing servers for the Spreadsheet::HTML smoke tests. Compare those to the newer smoke test servers that pass and we see that yes indeed, the failures were coming from servers that had HTML::Entities v1.27 installed while v3.69 (the current version at the time of this publishing) was installed on virtually all that passed. Enter perlbrew (easily manage versions of Perl) and cpanm (easily install and uninstall modules for those versions). From here it is not hard to recreate the failure: just
find the older version of HTML::Entities and replace the one being used with it. Unfortunately that requires figuring out which HTML::Parser distro is hiding v1.27 ... the very first available contains v1.13, maybe this is sufficient enough:
|
|
$ cpanm -n http://backpan.perl.org/authors/id/G/GA/GAAS/HTML-Parser-2.14.tar.gz $ cpanm -v Spreadsheet::HTML ... t/00-load.t ........... ok t/01-get-data.t ....... ok t/02-cache.t .......... ok t/03-encode-data.t .... 1/12 # Failed test 'encoding all HTML entities' # at t/03-encode-data.t line 42. # got: '<table><tr><th><</th><th>=</th><th>&</th><th>></th><th>"</th><th>'</th></tr><tr><td><</td><td>=</td><td>&</td><td>></td><td>"</td><td>'</td></tr></table>' # expected: '<table><tr><th><</th><th>=</th><th>&</th><th>></th><th>"</th><th>'</th></tr><tr><td><</td><td>=</td><td>&</td><td>></td><td>"</td><td>'</td></tr></table>' # Looks like you failed 1 test of 12. t/03-encode-data.t .... Dubious, test returned 1 (wstat 256, 0x100) Failed 1/12 subtests |
Bingo! Failure recreated. And now to finish our test: install the latest HTML::Entities and see if that clears up the error:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 |
$ cpanm HTML::Entities --> Working on HTML::Entities Fetching http://www.cpan.org/authors/id/G/GA/GAAS/HTML-Parser-3.71.tar.gz ... OK Configuring HTML-Parser-3.71 ... OK ==> Found dependencies: HTML::Tagset --> Working on HTML::Tagset Fetching http://www.cpan.org/authors/id/P/PE/PETDANCE/HTML-Tagset-3.20.tar.gz ... OK Configuring HTML-Tagset-3.20 ... OK Building and testing HTML-Tagset-3.20 ... OK Successfully installed HTML-Tagset-3.20 Building and testing HTML-Parser-3.71 ... OK Successfully installed HTML-Parser-3.71 (upgraded from 1.13) 2 distributions installed $ cpanm Spreadsheet::HTML --> Working on Spreadsheet::HTML Fetching http://www.cpan.org/authors/id/J/JE/JEFFA/Spreadsheet-HTML-0.34.tar.gz ... OK Configuring Spreadsheet-HTML-0.34 ... OK Building and testing Spreadsheet-HTML-0.34 ... OK Successfully installed Spreadsheet-HTML-0.34 1 distribution installed |
The solution will be to specify some version of HTML::Entities that is older than v1.27 but not necessarily the current one (v3.69) in the Makefile for HTML::AutoTag. I will start by requiring v3.69 and in the unlikely event that some server cannot use it (highly unlikely) then i can fall back on some older version. I am quite confident v3.69 will work fine for all, however. A new version of HTML::AutoTag will be uploaded the PAUSE, and after a few days a new version of Spreadsheet::HTML that specifies that version of HTML::AutoTag will also be uploaded.
The lesson here is not necessarily to always specify certain versions in your dependencies, but to pick and choose the right time to solve bugs that have minor consequence. 2 fails versus 115 passes and the error is within a very minor feature set -- what's the hurry? 🙂