/[webpac2]/trunk/lib/WebPAC/Input
This is repository of my old source code which isn't updated any more. Go to git.rot13.org for current projects!
ViewVC logotype

Log of /trunk/lib/WebPAC/Input

View Directory Listing Directory Listing


Sticky Revision:

Revision 1366 - Directory Listing
Modified Thu Dec 15 21:40:02 2011 UTC (12 years, 4 months ago) by dpavlin
import OAI repository

Revision 1365 - Directory Listing
Modified Wed May 4 13:44:07 2011 UTC (12 years, 11 months ago) by dpavlin
specify KOHA_DSN

Revision 1361 - Directory Listing
Modified Wed Mar 16 20:16:42 2011 UTC (13 years, 1 month ago) by dpavlin
added sqlite_unicode to force utf-8 from SQLite

Revision 1360 - Directory Listing
Modified Tue Mar 15 22:50:20 2011 UTC (13 years, 1 month ago) by dpavlin
we are not ignoring $mfn

Revision 1359 - Directory Listing
Modified Tue Mar 15 22:47:25 2011 UTC (13 years, 1 month ago) by dpavlin
read data from RDBMS using DBI


Revision 1349 - Directory Listing
Modified Sat Oct 16 18:25:59 2010 UTC (13 years, 6 months ago) by dpavlin
fix \N NULL skipping

Revision 1347 - Directory Listing
Modified Sat Oct 16 17:51:00 2010 UTC (13 years, 6 months ago) by dpavlin
chomp lines

Revision 1346 - Directory Listing
Modified Sat Oct 16 17:47:25 2010 UTC (13 years, 6 months ago) by dpavlin
ignore \N fields (NULL)

Revision 1343 - Directory Listing
Modified Sat Oct 16 17:38:47 2010 UTC (13 years, 6 months ago) by dpavlin
tab separated values input module


Revision 1327 - Directory Listing
Modified Tue Feb 9 20:14:23 2010 UTC (14 years, 2 months ago) by dpavlin
correctly decode utf-8 from marc files


Revision 1326 - Directory Listing
Modified Tue Feb 2 19:30:48 2010 UTC (14 years, 2 months ago) by dpavlin
log errors


Revision 1325 - Directory Listing
Modified Tue Feb 2 18:03:17 2010 UTC (14 years, 2 months ago) by dpavlin
added offset and limit [0.02]


Revision 1322 - Directory Listing
Modified Tue Jan 26 14:16:36 2010 UTC (14 years, 2 months ago) by dpavlin
Merge remote branch 'remotes/Sack'

Conflicts:
	bin/dump2marc.pl

Revision 1305 - Directory Listing
Modified Sun Sep 20 22:26:27 2009 UTC (14 years, 7 months ago) by dpavlin
implement experimental (and probably broken) low-level offset and limit


Revision 1303 - Directory Listing
Modified Sun Sep 20 19:56:33 2009 UTC (14 years, 7 months ago) by dpavlin
implement efficiant offset


Revision 1302 - Directory Listing
Modified Sun Sep 20 19:05:56 2009 UTC (14 years, 7 months ago) by dpavlin
skip empty lines and nicer output


Revision 1296 - Directory Listing
Modified Sat Sep 19 23:29:23 2009 UTC (14 years, 7 months ago) by dpavlin
- added support for Direct Export format to existing BRS/Tagged
- specify file glob (as from ovid-download-results.pl) for multiple files


Revision 1288 - Directory Listing
Modified Sat Sep 19 08:41:06 2009 UTC (14 years, 7 months ago) by dpavlin
join PA and JI

Revision 1287 - Directory Listing
Modified Fri Sep 18 21:38:09 2009 UTC (14 years, 7 months ago) by dpavlin
- join FU and FX fields, split ID SC and DE on ;
- implemented limit and offset for speedup


Revision 1276 - Directory Listing
Modified Wed Aug 19 16:07:39 2009 UTC (14 years, 8 months ago) by dpavlin
added sane defaults if not specified in configuration file


Revision 1254 - Directory Listing
Modified Mon Jul 27 16:24:41 2009 UTC (14 years, 8 months ago) by dpavlin
force utf-8 encoding on all data comming from file

Revision 1244 - Directory Listing
Modified Mon Jul 20 22:00:43 2009 UTC (14 years, 9 months ago) by dpavlin
don't fallback to input name, but use first sheet instead


Revision 1234 - Directory Listing
Modified Fri Jul 10 13:53:28 2009 UTC (14 years, 9 months ago) by dpavlin
don't overwrite cache marc file


Revision 1232 - Directory Listing
Modified Thu Jul 9 22:01:11 2009 UTC (14 years, 9 months ago) by dpavlin
dump items without marc instead of dieing


Revision 1231 - Directory Listing
Modified Thu Jul 9 17:00:51 2009 UTC (14 years, 9 months ago) by dpavlin
fetch MARC records directly from Koha database

Just create local file out of them, they need to be
converted in hash to be really useful inside WebPAC


Revision 1222 - Directory Listing
Modified Tue Jun 9 21:41:12 2009 UTC (14 years, 10 months ago) by dpavlin
- version bump [0.06]
- produce column names and labels for vhost/webpac2.cgi
- implement normalize callback which put Excel data into _rows and search values


Revision 1217 - Directory Listing
Modified Tue Jun 9 18:38:36 2009 UTC (14 years, 10 months ago) by dpavlin
select sheet: worksheet param, name of input, first one


Revision 1215 - Directory Listing
Modified Tue Jun 2 13:16:02 2009 UTC (14 years, 10 months ago) by dpavlin
more fields to join (from multi-line input)


Revision 1209 - Directory Listing
Modified Sat May 30 14:21:58 2009 UTC (14 years, 10 months ago) by dpavlin
 r1904@llin:  dpavlin | 2009-05-30 16:21:56 +0200
 better support for DOI in ISI data export


Revision 1194 - Directory Listing
Modified Wed May 27 09:31:35 2009 UTC (14 years, 10 months ago) by dpavlin
 r1878@llin:  dpavlin | 2009-05-27 11:31:28 +0200
 CR field in ISI format now also contains 'full' line from original
 file as well as parsed components


Revision 1186 - Directory Listing
Modified Tue May 19 14:46:12 2009 UTC (14 years, 11 months ago) by dpavlin
added WebPAC::Input::CSV

Revision 1130 - Directory Listing
Modified Tue Apr 21 21:06:29 2009 UTC (15 years ago) by dpavlin
 r1768@llin:  dpavlin | 2009-04-21 23:04:32 +0200
 hush debug output


Revision 1126 - Directory Listing
Modified Mon Apr 20 13:44:40 2009 UTC (15 years ago) by dpavlin
 r1760@llin:  dpavlin | 2009-04-20 15:44:39 +0200
 tweak implementation to actually work


Revision 1124 - Directory Listing
Modified Sun Apr 19 23:35:31 2009 UTC (15 years ago) by dpavlin
 r1756@llin:  dpavlin | 2009-04-20 01:35:20 +0200
 added WebPAC::Input::Ovid for citation format


Revision 1110 - Directory Listing
Modified Sat Sep 6 10:53:47 2008 UTC (15 years, 7 months ago) by dpavlin
- version dependency on MARC::Fast is handled by Makefile.PL
- include_subfields in returned hash


Revision 1105 - Directory Listing
Modified Mon Aug 4 19:35:23 2008 UTC (15 years, 8 months ago) by dpavlin
 r1724@llin:  dpavlin | 2008-08-04 21:35:00 +0200
 fix input/excel encoding problems


Revision 1100 - Directory Listing
Modified Sat Aug 2 23:46:41 2008 UTC (15 years, 8 months ago) by dpavlin
Make cleanup of encodings, moving webpac closer to having
internal utf-8 representation.

This will break current code, but is really neceserry
step toward checking input encoding for validity


Revision 1089 - Directory Listing
Modified Mon Jan 28 18:30:07 2008 UTC (16 years, 2 months ago) by dpavlin
 r1702@llin:  dpavlin | 2008-01-28 19:29:41 +0100
 EBSCO text file export support


Revision 1077 - Directory Listing
Modified Wed Nov 28 22:52:01 2007 UTC (16 years, 4 months ago) by dpavlin
fake mfn to make rest of WebPAC happy


Revision 1063 - Directory Listing
Modified Tue Nov 27 21:01:44 2007 UTC (16 years, 4 months ago) by dpavlin
pod fixes


Revision 1057 - Directory Listing
Modified Tue Nov 20 10:08:02 2007 UTC (16 years, 5 months ago) by dpavlin
 r1650@llin:  dpavlin | 2007-11-20 11:07:57 +0100
 final tweaks for WebPAC::Input::PDF, emit fields A .. ZZ


Revision 1055 - Directory Listing
Modified Tue Nov 20 09:30:58 2007 UTC (16 years, 5 months ago) by dpavlin
 r1646@llin:  dpavlin | 2007-11-20 10:30:52 +0100
 document column names


Revision 1054 - Directory Listing
Modified Tue Nov 20 09:30:56 2007 UTC (16 years, 5 months ago) by dpavlin
 r1645@llin:  dpavlin | 2007-11-19 23:05:23 +0100
 added experimenal (still not working) WebPAC::Input::PDF


Revision 998 - Directory Listing
Modified Sun Nov 4 16:47:03 2007 UTC (16 years, 5 months ago) by dpavlin
 r1536@llin:  dpavlin | 2007-11-04 17:47:03 +0100
 better handle invalid XML files


Revision 992 - Directory Listing
Modified Sun Nov 4 13:47:02 2007 UTC (16 years, 5 months ago) by dpavlin
 r1523@llin:  dpavlin | 2007-11-04 14:39:39 +0100
 hush all kind of debugging output


Revision 989 - Directory Listing
Modified Sun Nov 4 13:26:06 2007 UTC (16 years, 5 months ago) by dpavlin
 r1517@llin:  dpavlin | 2007-11-04 14:26:05 +0100
 New rewamp of WebPAC::Input::XML with added mungle rules (perl code really)
 to modify xml hash returned from XML::Simple


Revision 984 - Directory Listing
Modified Sun Nov 4 11:17:21 2007 UTC (16 years, 5 months ago) by dpavlin
 r1507@llin:  dpavlin | 2007-11-04 12:17:19 +0100
 sort XML files by filename before processing it (to preserve order if
 filenames are sortable and they usually are)


Revision 981 - Directory Listing
Modified Sat Nov 3 13:33:21 2007 UTC (16 years, 5 months ago) by dpavlin
 r1500@llin:  dpavlin | 2007-11-03 14:32:06 +0100
 tweak


Revision 970 - Directory Listing
Modified Fri Nov 2 14:29:11 2007 UTC (16 years, 5 months ago) by dpavlin
 r1483@llin:  dpavlin | 2007-11-02 15:29:09 +0100
 return parsed XML hash


Revision 968 - Directory Listing
Modified Fri Nov 2 13:59:10 2007 UTC (16 years, 5 months ago) by dpavlin
 r1479@llin:  dpavlin | 2007-11-02 14:59:05 +0100
 begin work on WebPAC::Input::XML


Revision 908 - Directory Listing
Modified Mon Oct 29 23:20:13 2007 UTC (16 years, 5 months ago) by dpavlin
leader from WebPAC::Input::MARC is now available as rec('leader')

for mondifications within leader, use substr(rec('leader'),from,to)
instead of proposed leader(field,nr) syntax


Revision 904 - Directory Listing
Modified Fri Oct 12 12:07:35 2007 UTC (16 years, 6 months ago) by dpavlin
 r1355@llin:  dpavlin | 2007-10-12 14:07:29 +0200
 fix empty tags


Revision 902 - Directory Listing
Modified Wed Oct 10 21:00:27 2007 UTC (16 years, 6 months ago) by dpavlin
more tags to join


Revision 901 - Directory Listing
Modified Wed Oct 10 20:05:45 2007 UTC (16 years, 6 months ago) by dpavlin
added splitting of tags into subfields (CR for now)
and ability to join tags into single line (AB)


Revision 900 - Directory Listing
Modified Wed Oct 10 19:46:58 2007 UTC (16 years, 6 months ago) by dpavlin
 r1348@llin:  dpavlin | 2007-10-10 21:46:55 +0200
 added URL to some documentation


Revision 899 - Directory Listing
Modified Wed Oct 10 19:01:57 2007 UTC (16 years, 6 months ago) by dpavlin
 r1345@llin:  dpavlin | 2007-10-10 21:01:02 +0200
 and working parser


Revision 898 - Directory Listing
Modified Wed Oct 10 19:01:55 2007 UTC (16 years, 6 months ago) by dpavlin
 r1344@llin:  dpavlin | 2007-10-10 20:34:20 +0200
 skeleton of support for ISI export format, parse headers


Revision 894 - Directory Listing
Modified Sun Oct 7 22:51:57 2007 UTC (16 years, 6 months ago) by dpavlin
 r1334@llin:  dpavlin | 2007-10-08 00:51:58 +0200
 typo


Revision 888 - Directory Listing
Modified Mon Sep 3 15:28:33 2007 UTC (16 years, 7 months ago) by dpavlin
 r1325@llin:  dpavlin | 2007-09-03 17:28:32 +0200
 fix warning


Revision 873 - Directory Listing
Modified Fri Jun 22 00:03:46 2007 UTC (16 years, 10 months ago) by dpavlin
 r1298@llin:  dpavlin | 2007-06-22 02:03:23 +0200
 input_config can be given to new or open now


Revision 871 - Directory Listing
Modified Thu Jun 21 23:54:42 2007 UTC (16 years, 10 months ago) by dpavlin
 r1294@llin:  dpavlin | 2007-06-22 01:54:51 +0200
 extract common _to_hash into WebPAC::Input::Helper


Revision 870 - Directory Listing
Modified Thu Jun 21 23:54:41 2007 UTC (16 years, 10 months ago) by dpavlin
 r1293@llin:  dpavlin | 2007-06-22 01:46:20 +0200
 finish dbf input


Revision 869 - Directory Listing
Modified Thu Jun 21 21:26:19 2007 UTC (16 years, 10 months ago) by dpavlin
 r1290@llin:  dpavlin | 2007-06-21 23:26:25 +0200
 experimental (still unfinished) dbf input


Revision 825 - Directory Listing
Modified Fri May 18 21:41:19 2007 UTC (16 years, 11 months ago) by dpavlin
request Biblio::Isis 0.24 to ignore empty subfields


Revision 798 - Directory Listing
Modified Sun Feb 4 13:31:38 2007 UTC (17 years, 2 months ago) by dpavlin
store filter (no tests for it, though!)


Revision 797 - Directory Listing
Modified Sun Feb 4 13:28:30 2007 UTC (17 years, 2 months ago) by dpavlin
finish tweaking mock framework, test and fix problem with slashes in modify_record


Revision 796 - Directory Listing
Modified Sun Feb 4 12:42:43 2007 UTC (17 years, 2 months ago) by dpavlin
a try at mocking of inputs in WebPAC::Input::Test


Revision 779 - Directory Listing
Modified Sun Nov 5 14:52:04 2006 UTC (17 years, 5 months ago) by dpavlin
 r1137@llin:  dpavlin | 2006-11-05 15:51:19 +0100
 no need to have MFN twice in record (it is also added by _to_hash)


Revision 778 - Directory Listing
Modified Sun Nov 5 14:51:59 2006 UTC (17 years, 5 months ago) by dpavlin
 r1136@llin:  dpavlin | 2006-11-05 15:49:50 +0100
 debug shouldn't auto-vivify all fields!


Revision 777 - Directory Listing
Modified Sun Nov 5 14:48:12 2006 UTC (17 years, 5 months ago) by dpavlin
 r1133@llin:  dpavlin | 2006-11-05 15:48:00 +0100
 first cut at getting Project Gutenberg's RDF as input format for WebPAC


Revision 774 - Directory Listing
Modified Fri Nov 3 20:56:21 2006 UTC (17 years, 5 months ago) by dpavlin
another swiping API change: input->dump is gone, replaced
with input->dump_ascii which is more understandable.
If you want to override default behaviour
(which is to use Data::Dump's dump in input->fetch_rec)
define dump_ascii in low-level WebPAC::Input:: API


Revision 772 - Directory Listing
Modified Fri Nov 3 20:40:38 2006 UTC (17 years, 5 months ago) by dpavlin
 r1124@llin:  dpavlin | 2006-11-03 21:39:00 +0100
 use MARC::Fast 0.05 to_ascii to implement dump_rec


Revision 770 - Directory Listing
Modified Fri Nov 3 20:21:14 2006 UTC (17 years, 5 months ago) by dpavlin
 r1120@llin:  dpavlin | 2006-11-03 21:22:05 +0100
 pod fix


Revision 728 - Directory Listing
Modified Fri Sep 29 19:52:26 2006 UTC (17 years, 6 months ago) by dpavlin
 r1047@llin:  dpavlin | 2006-09-29 21:49:48 +0200
 move to new low-level API


Revision 726 - Directory Listing
Modified Fri Sep 29 19:52:17 2006 UTC (17 years, 6 months ago) by dpavlin
 r1045@llin:  dpavlin | 2006-09-29 21:38:42 +0200
 change low-level API to be OO (and remove various ugly cludges).


Revision 652 - Directory Listing
Modified Thu Sep 7 15:01:45 2006 UTC (17 years, 7 months ago) by dpavlin
refactored internal WebPAC::Input::* API a bit, added dump_rec,
validate is now more clever and reports all errors from database at end


Revision 625 - Directory Listing
Modified Sat Aug 26 12:00:36 2006 UTC (17 years, 7 months ago) by dpavlin
 r878@llin:  dpavlin | 2006-08-26 14:00:08 +0200
 removed some debugging output (or moved it to debug level), few tweaks [2.26]


Revision 623 - Directory Listing
Modified Sat Aug 26 12:00:25 2006 UTC (17 years, 7 months ago) by dpavlin
 r876@llin:  dpavlin | 2006-08-25 21:53:19 +0200
 remove OpenIsis support (it was broken for quite some time), make hash_filter chatty (debugging)


Revision 619 - Directory Listing
Modified Fri Aug 25 12:31:06 2006 UTC (17 years, 7 months ago) by dpavlin
 r867@llin:  dpavlin | 2006-08-25 14:32:05 +0200
 statistics now show data before modify_records


Revision 615 - Directory Listing
Modified Wed Aug 23 14:28:48 2006 UTC (17 years, 8 months ago) by dpavlin
added include_subfields needed for marc_original_order


Revision 597 - Directory Listing
Modified Thu Jul 13 11:54:33 2006 UTC (17 years, 9 months ago) by dpavlin
 r831@llin:  dpavlin | 2006-07-13 13:56:19 +0200
 first cut in implementing modify_records using automatically generated regexpes


Revision 524 - Directory Listing
Modified Sun May 21 19:38:56 2006 UTC (17 years, 11 months ago) by dpavlin
added from and to parametars for start and end row to import


Revision 521 - Directory Listing
Modified Thu May 18 13:49:08 2006 UTC (17 years, 11 months ago) by dpavlin
 r691@llin:  dpavlin | 2006-05-18 15:52:34 +0200
 store MFN (line number, really) correctly in field 000


Revision 498 - Directory Listing
Modified Sun May 14 19:45:45 2006 UTC (17 years, 11 months ago) by dpavlin
 r653@llin:  dpavlin | 2006-05-14 21:48:48 +0200
 added Excel input format


Revision 497 - Directory Listing
Modified Sun May 14 19:45:36 2006 UTC (17 years, 11 months ago) by dpavlin
 r652@llin:  dpavlin | 2006-05-14 21:47:38 +0200
 documentation fix for open_db


Revision 416 - Directory Listing
Modified Sun Feb 26 23:21:50 2006 UTC (18 years, 1 month ago) by dpavlin
 r494@llin:  dpavlin | 2006-02-27 00:22:59 +0100
 implemented recode option to input (for now, just for MARC)


Revision 337 - Directory Listing
Modified Sat Dec 31 16:41:35 2005 UTC (18 years, 3 months ago) by dpavlin
 r343@llin:  dpavlin | 2005-12-31 17:44:58 +0100
 fix clash of $self->{size} from WebPAC::Input and WebPAC::Input::MARC,
 dokument _marc_size property


Revision 336 - Directory Listing
Modified Sat Dec 31 16:28:18 2005 UTC (18 years, 3 months ago) by dpavlin
 r341@llin:  dpavlin | 2005-12-31 17:31:33 +0100
 fix possible corruption of fields < 100


Revision 309 - Directory Listing
Modified Tue Dec 20 19:01:27 2005 UTC (18 years, 4 months ago) by dpavlin
 r336@athlon:  dpavlin | 2005-12-20 20:02:10 +0100
 use to_hash from MARC::Fast, not fetch... pfff!


Revision 307 - Directory Listing
Modified Tue Dec 20 00:03:04 2005 UTC (18 years, 4 months ago) by dpavlin
moved clean into WebPAC::Output::Estraier, cleanup


Revision 298 - Directory Listing
Modified Mon Dec 19 19:55:21 2005 UTC (18 years, 4 months ago) by dpavlin
 r317@athlon:  dpavlin | 2005-12-19 20:56:26 +0100
 some fixes and cleanup, moved module versions to Makefile.PL


Revision 291 - Directory Listing
Modified Sun Dec 18 23:34:24 2005 UTC (18 years, 4 months ago) by dpavlin
 r11789@llin:  dpavlin | 2005-12-19 06:29:24 +0100
 final tweaks, version bumping [2.00_6]


Revision 290 - Directory Listing
Modified Sun Dec 18 23:10:02 2005 UTC (18 years, 4 months ago) by dpavlin
 r11787@llin:  dpavlin | 2005-12-19 06:10:47 +0100
 MARC indexing seems to work


Revision 289 - Directory Listing
Modified Sun Dec 18 22:16:44 2005 UTC (18 years, 4 months ago) by dpavlin
 r11784@llin:  dpavlin | 2005-12-19 05:17:24 +0100
 don't use Exporter after all


Revision 286 - Directory Listing
Modified Sun Dec 18 21:06:46 2005 UTC (18 years, 4 months ago) by dpavlin
 r11778@llin:  dpavlin | 2005-12-19 03:59:54 +0100
 move work on input


Revision 285 - Directory Listing
Modified Sun Dec 18 21:06:39 2005 UTC (18 years, 4 months ago) by dpavlin
 r11777@llin:  dpavlin | 2005-12-19 00:02:47 +0100
 refactor Input::ISIS::* [0.02]


Revision 265 - Directory Listing
Modified Fri Dec 16 16:23:44 2005 UTC (18 years, 4 months ago) by dpavlin
 r11736@llin:  dpavlin | 2005-12-16 21:22:26 +0100
 die if database can't be opened, confirms to test


Revision 251 - Directory Listing
Modified Thu Dec 15 14:12:00 2005 UTC (18 years, 4 months ago) by dpavlin
various updates to make lookups work (but they don't still)


Revision 113 - Directory Listing
Modified Wed Nov 23 00:14:05 2005 UTC (18 years, 5 months ago) by dpavlin
 r9064@llin:  dpavlin | 2005-11-23 01:15:24 +0100
 minor tweak for database routines, run.pl now iterates through all entries
 (to fix problem with stopping at first deleted entry)


Revision 21 - Directory Listing
Modified Sun Jul 17 22:28:11 2005 UTC (18 years, 9 months ago) by dpavlin
fixed ISIS size


Revision 10 - Directory Listing
Modified Sat Jul 16 20:35:30 2005 UTC (18 years, 9 months ago) by dpavlin
ISIS input is finished, low_mem option has code (and not only documentation :-)


Revision 9 - Directory Listing
Modified Sat Jul 16 17:14:43 2005 UTC (18 years, 9 months ago) by dpavlin
a bit more work on WebPAC::Input::ISIS


Revision 8 - Directory Listing
Modified Sat Jul 16 16:48:35 2005 UTC (18 years, 9 months ago) by dpavlin
little cleanup and first cut into WebPAC::Normalize::XML


Revision 6 - Directory Listing
Added Sat Jul 16 14:44:38 2005 UTC (18 years, 9 months ago) by dpavlin
added WebPAC::Input::ISIS


  ViewVC Help
Powered by ViewVC 1.1.26