Log of /lib/Grep/Source.pm

---

Revision 110 - (view) (annotate) - [select for diffs]
Modified Wed Mar 14 20:02:19 2007 UTC (16 years, 8 months ago) by dpavlin
File length: 7281 byte(s)
Diff to previous 109

another bunch of various tweaks, but Lucene still doesn't lock index right

---

Revision 109 - (view) (annotate) - [select for diffs]
Modified Wed Mar 14 18:46:37 2007 UTC (16 years, 8 months ago) by dpavlin
File length: 7177 byte(s)
Diff to previous 103

rewrite Grep::Search to be isa Jifty::Object

---

Revision 103 - (view) (annotate) - [select for diffs]
Modified Sun Mar 4 22:51:01 2007 UTC (16 years, 9 months ago) by dpavlin
File length: 7156 byte(s)
Diff to previous 102

fetch maximum of 15 pages from remote wiki when scraping results

---

Revision 102 - (view) (annotate) - [select for diffs]
Modified Sun Mar 4 22:16:23 2007 UTC (16 years, 9 months ago) by dpavlin
File length: 7155 byte(s)
Diff to previous 100

removed all debug warn(s) or move them to $self->log->debug

---

Revision 100 - (view) (annotate) - [select for diffs]
Modified Sat Feb 24 12:32:31 2007 UTC (16 years, 9 months ago) by dpavlin
File length: 7094 byte(s)
Diff to previous 96

isa Jifty::Object

---

Revision 96 - (view) (annotate) - [select for diffs]
Modified Sat Feb 24 11:56:18 2007 UTC (16 years, 9 months ago) by dpavlin
File length: 7024 byte(s)
Diff to previous 92

explicitly destroy $parent passed to plugins as another try to get around
Lucene's locking problems
use HTML::ResolveLink to resolve all links before add_record

---

Revision 92 - (view) (annotate) - [select for diffs]
Modified Sat Feb 24 11:16:05 2007 UTC (16 years, 9 months ago) by dpavlin
File length: 6793 byte(s)
Diff to previous 88

redirect errors and warnings to warn so they are all non-fatal

---

Revision 88 - (view) (annotate) - [select for diffs]
Modified Fri Feb 23 21:52:29 2007 UTC (16 years, 9 months ago) by dpavlin
File length: 6635 byte(s)
Diff to previous 86

treat missing results div as no results and don't die

---

Revision 86 - (view) (annotate) - [select for diffs]
Modified Fri Feb 23 21:16:44 2007 UTC (16 years, 9 months ago) by dpavlin
File length: 6619 byte(s)
Diff to previous 85

added hooks to Grep::Source->save to keep useful snippets of html in /tmp/grep (if writable)

---

Revision 85 - (view) (annotate) - [select for diffs]
Modified Fri Feb 23 20:47:08 2007 UTC (16 years, 9 months ago) by dpavlin
File length: 6446 byte(s)
Diff to previous 82

refactor most of code for scraping into common code making plugins really simple

---

Revision 82 - (view) (annotate) - [select for diffs]
Modified Fri Feb 23 18:10:26 2007 UTC (16 years, 9 months ago) by dpavlin
File length: 3498 byte(s)
Diff to previous 74

really work like designed as opposed to returning first available plugin (ouch!)

---

Revision 74 - (view) (annotate) - [select for diffs]
Modified Fri Feb 23 17:16:33 2007 UTC (16 years, 9 months ago) by dpavlin
File length: 3260 byte(s)
Diff to previous 73

fix warning

---

Revision 73 - (view) (annotate) - [select for diffs]
Modified Fri Feb 23 11:48:39 2007 UTC (16 years, 9 months ago) by dpavlin
File length: 3251 byte(s)
Diff to previous 72

each feed now has default source class which is called for it. Added PhpWiki
source. Code still has problems with Lucene locking.

---

Revision 72 - (view) (annotate) - [select for diffs]
Added Fri Feb 23 09:54:28 2007 UTC (16 years, 9 months ago) by dpavlin
File length: 2562 byte(s)

another great refactoring: added new Source object which implements
searching within feed (which now can be anything as long as it produce fields
which somewhat resamble RSS feed). Source plugins implement just (site or
source format specific) fetching of items. 

Sample implementation of MoinMoin scraper, which fetch full pages from wiki
for results, so it has performance impact on remote wiki, be kind to it.