cairo-dock-team team mailing list archive
-
cairo-dock-team team
-
Mailing list archive
-
Message #01882
[Merge] lp:~eduardo-mucelli/cairo-dock-plug-ins-extras/WebSearch into lp:cairo-dock-plug-ins-extras
Eduardo Mucelli R. Oliveira has proposed merging lp:~eduardo-mucelli/cairo-dock-plug-ins-extras/WebSearch into lp:cairo-dock-plug-ins-extras.
Requested reviews:
Cairo-Dock Team (cairo-dock-team)
WebSearch now fetch results from Twitter. Fixing a bug that I introduced in the code refactoring, forget that some engines use page, and some use offset. Modularizing the engines. Shortening the description the same way as the url because some tweets are very long. Engine class now uses a neat way for lazy-loading and engine instantiation.
--
https://code.launchpad.net/~eduardo-mucelli/cairo-dock-plug-ins-extras/WebSearch/+merge/25810
Your team Cairo-Dock Team is requested to review the proposed merge of lp:~eduardo-mucelli/cairo-dock-plug-ins-extras/WebSearch into lp:cairo-dock-plug-ins-extras.
=== modified file 'WebSearch/Changelog.txt'
--- WebSearch/Changelog.txt 2010-05-18 14:29:46 +0000
+++ WebSearch/Changelog.txt 2010-05-22 02:22:22 +0000
@@ -1,3 +1,4 @@
+1.2.0: (May/21/2010): WebSearch now fetch results from Twitter. Fixing a bug that I introduced in the code refactoring, forget that some engines use page, and some use offset. Modularizing the engines. Shortening the description the same way as the url because some tweets are very long. Engine class now uses a neat way for lazy-loading and engine instantiation.
1.1.0: (May/17/2010): WebSearch now fetch results from ImageShack. Sub-menu is using icons to a better engine indication.
1.0.0: (May/12/2010): A heavy code refactoring, each search engine has its own module. Webshots has changed its HTML the whole fetching was recoded. Removing user-defined option for a pre-fetching limit. Finally a totally functional parallelized thumbnail download.
0.7.3: (May/10/2010): Fixing the Google search since the engine changed the result stats HTML. Fixing the Bing result stats. Fixing the stats showing.
=== modified file 'WebSearch/WebSearch'
--- WebSearch/WebSearch 2010-05-18 13:47:31 +0000
+++ WebSearch/WebSearch 2010-05-22 02:22:22 +0000
@@ -16,14 +16,14 @@
# GNU General Public License for more details.
# This applet provides an interface to some search engines such as
-# Google, Bing, Teoma, Yahoo!, Youtube, Webshots, Flickr, Wikipedia, and ImageShack.
-# To choose the search engine you can
-# (1) Right-click -> Configure this applet -> Configuration (tab) -> Search engine
-# (2) Scroll up or down over the icon (applicable only for the first search)
-# You can search in three ways
+# Google, Bing, Teoma, Yahoo!, Youtube, Webshots, Flickr, Wikipedia, ImageShack, and Twitter.
+# To choose the search engine you can
+# (1) Right-click on the main icon -> WebSearch -> (Choose the engine)
+# (2) Right-click -> Configure this applet -> Configuration -> Search engine
+# (3) Scroll up or down over the icon (applicable only for the first search)
+# You can search in two ways
# (1) Middle-click on the main icon
-# (2) Right-click on the main icon -> WebSearch -> (Choose the engine)
-# (3) Left-click on main icon (applicable only for the first search)
+# (2) Left-click on main icon (right after choosing a new engine)
# Type your query and validate. Each result will be shown as an sub-icon.
# Left-click to open the the result in the default web browser
# Middle-click on the sub-icon of any result to show its description
@@ -39,9 +39,7 @@
end
class String
- def to_b
- "true" == downcase # string to boolean
- end
+ def to_b;"true" == downcase;end # string to boolean
end
module WebSearch
@@ -61,7 +59,7 @@
applet_sub_icons_object = applet_service.object("#{applet_path}/sub_icons")
applet_sub_icons_object.introspect
- applet_sub_icons_object.default_iface = "org.cairodock.CairoDock.subapplet" # list of icons contained in our sub-dock, or in our desklet
+ applet_sub_icons_object.default_iface = 'org.cairodock.CairoDock.subapplet' # list of icons contained in our sub-dock, or in our desklet
applet = Applet.new applet_object, applet_sub_icons_object
applet.start
@@ -73,24 +71,23 @@
class Applet
require './lib/Engine.rb'
- # require './lib/Exceptions.rb'
- attr_accessor :engine, :query, :engines,
+ attr_accessor :engine, :engines, :query,
:number_of_fetched_links, :number_of_displayed_links, :page_of_displayed_links,
:show_current_page, :show_description_instead_url, :show_thumbnail_preview,
- :scroll_engine_index, :add_menu_items_available
+ :scroll_engine_index
- DialogActiveTime = 5 # time in seconds the dialog window will be active
+ DialogActiveTime = 5 # time in seconds that the dialog window will be active
def initialize applet, sub_icons
self.query = ""
- self.engines = %w(Google Bing Yahoo! Teoma Wikipedia Youtube Webshots Flickr ImageShack)
+ self.engines = Engines::List # ./lib/Engines.rb
self.engine = Engine.new
self.scroll_engine_index = 0 # current index when scrolling through search engines
@icon = applet
@sub_icons = sub_icons
reset_search_settings
- set_configuration_parameters # setting the self.configuration content
+ set_configuration_parameters
end
def set_configuration_parameters
@@ -109,6 +106,7 @@
verify_user_action
end
+ # Signal handling
def verify_user_action
@icon.on_signal("on_build_menu") do |param| # right click signal
action_on_build_menu
@@ -167,13 +165,18 @@
unless answer.empty?
reset_search_settings unless self.query.empty?
self.query = answer
- self.engine = self.engine.connect # only when the fetch is imminent the engine connection occurs
- fetch_next_resulting_page
+ begin
+ self.engine = self.engine.connect # only when the fetch is imminent the engine connection occurs
+ rescue Exceptions::UnknownEngineException => e
+ WebSearch.log e.message
+ else
+ fetch_next_resulting_page
+ end
end
end
def reset_search_settings
- self.engine.links = []
+ self.engine.links =[] # .clear cant be used yet
self.engine.stats = ""
self.page_of_displayed_links = 0 # current pagination of displayed links
Link.reset_next_id
@@ -212,7 +215,7 @@
set_configuration_parameters if config_has_changed
end
- # Scrolling behavior can be switch the search engine, or fetch another resulting page
+ # Scrolling can switch the search engine, or fetch another resulting page
def action_on_scroll scroll_up
if self.query.empty? # before the first query it is possible scroll through engines
if scroll_up
@@ -241,7 +244,12 @@
offset = self.page_of_displayed_links * self.number_of_displayed_links # the position of the first link in the self.engine.links array
if self.engine.links.size <= offset # user already scrolled by the fetched links, fetch more
inform_start_of_waiting_process
- self.engine.links = self.engine.retrieve_links(self.query, offset) # receive the fetched links
+ self.engine.links = case self.engine.name # receive the fetched links
+ # some engines use the concept of offset which is the first index of an interval of links/images to be shown
+ # but there is those that use a sequential page (1,2,3, ...) and each one has an amount of links/images, etc
+ when "Google", "Bing", "Yahoo", "Webshots", "Wikipedia"; self.engine.retrieve_links(self.query, offset)
+ when "Teoma","Youtube","Flickr","ImageShack","Twitter"; self.engine.retrieve_links(self.query, self.page_of_displayed_links + 1)
+ end
inform_end_of_waiting_process
end
self.page_of_displayed_links += 1 # sequential page identification, lets go to the next
@@ -252,9 +260,9 @@
# Since the previous results are already stored in self.engine.links, it is necessary just to
# select the its interval that starts with the first link of the previous page.
- # An easier approach would be query google again with page-1 but it would result
+ # An easier approach would be to query the engine again with page-1 but it would result
# more queries to the page, consequently it has some drawbacks such as increasing the
- # probability of Google block the mechanized access, more bandwith, etc.
+ # probability of forbidden mechanized access, more bandwith, etc.
def fetch_previous_resulting_page
if self.page_of_displayed_links > 1 # there is no previous page from the first one
self.page_of_displayed_links -= 1 # one page back
@@ -274,7 +282,9 @@
self.engine.links[inicio, self.number_of_displayed_links].each do |link| # first let's download the thumbs if necessary
if link.instance_of?(ThumbnailedLink) and not link.downloaded_thumb? # class that provides thumbs and a not yet downloaded thumb
if self.show_thumbnail_preview # user want to see thumbs, so let's get it
- threads << Thread.new {link.download_thumbnail}
+ threads << Thread.new {
+ link.download_thumbnail
+ }
end
end
end
=== modified file 'WebSearch/WebSearch.conf'
--- WebSearch/WebSearch.conf 2010-05-18 22:01:56 +0000
+++ WebSearch/WebSearch.conf 2010-05-22 02:22:22 +0000
@@ -1,4 +1,4 @@
-#!en;1.1.0
+#!en;1.2.0
#[gtk-about]
[Icon]
@@ -83,7 +83,7 @@
#[gtk-preferences]
[Configuration]
-#l[Google;Bing;Yahoo!;Teoma;Youtube;Webshots;Flickr;Wikipedia;ImageShack] Search engine :
+#l[Google;Bing;Yahoo!;Teoma;Youtube;Webshots;Flickr;Wikipedia;ImageShack;Twitter] Search engine :
engine = 0
#i[5;10] Maximum number of results shown :
#{in sub-icons.}
@@ -92,6 +92,6 @@
show current page = true
#b Show the description of the result instead of its URL in the sub-icons ?
show description instead url = true
-#b Enable thumbnail preview for Youtube, Webshots, and Flickr search ?
+#b Enable thumbnail preview for Youtube, Webshots, Flickr, and ImageShack searches ?
#{for slow connections, disable will result in significantly faster fetching.}
show thumbnail preview = true
=== modified file 'WebSearch/auto-load.conf'
--- WebSearch/auto-load.conf 2010-05-18 13:47:31 +0000
+++ WebSearch/auto-load.conf 2010-05-22 02:22:22 +0000
@@ -4,10 +4,10 @@
author = Eduardo Mucelli Rezende Oliveira
# A short description of the applet and how to use it.
-description = This applet provides an interface to some search engines such as\nGoogle, Bing, Teoma, Yahoo!, Youtube, Webshots, Flickr, Wikipedia, and ImageShack.\nTo choose the search engine you can\n (1) Right-click -> Configure this applet -> Configuration -> Search engine\n (2) Scroll up or down over the icon (applicable only for the first search)\nYou can search in three ways\n (1) Middle-click on the main icon\n (2) Right-click on the main icon -> WebSearch -> (Choose the engine)\n (3) Left-click on main icon (applicable only for the first search)\nType your query and validate. Each result will be shown as a sub-icon.\nLeft-click to open the the result in the default Web Browser\nMiddle-click on the sub-icon of any result to show its description\nScroll up to fetch the next results\nScroll down to fetch the previous results\nLeft-click on the main icon to show search stats
+description = This applet provides an interface to some search engines such as\nGoogle, Bing, Teoma, Yahoo!, Youtube, Webshots, Flickr, Wikipedia, ImageShack, and Twitter.\nTo choose the search engine you can\n (1) Right-click on the main icon -> WebSearch -> (Choose the engine)\n (2) Right-click -> Configure this applet -> Configuration -> Search engine\n (3) Scroll up or down over the icon (applicable only for the first search)\nYou can search in three ways\n (1) Middle-click on the main icon\n (2) Left-click on main icon (right after choosing a new engine)\nType your query and validate. Each result will be shown as a sub-icon.\nLeft-click to open the the result in the default Web Browser\nMiddle-click on the sub-icon of any result to show its description\nScroll up to fetch the next results\nScroll down to fetch the previous results\nLeft-click on the main icon to show search stats
# Category of the applet : 2 = accessory, 3 = Desktop, 4 = Controler
category = 2
# Version of the applet; change it everytime you change something in the config file. Don't forget to update the version both in this file and in the config file.
-version = 1.1.0
+version = 1.2.0
=== added file 'WebSearch/images/data/Twitter.png'
Binary files WebSearch/images/data/Twitter.png 1970-01-01 00:00:00 +0000 and WebSearch/images/data/Twitter.png 2010-05-22 02:22:22 +0000 differ
=== removed file 'WebSearch/images/data/Yahoo!.png'
Binary files WebSearch/images/data/Yahoo!.png 2010-05-18 13:47:51 +0000 and WebSearch/images/data/Yahoo!.png 1970-01-01 00:00:00 +0000 differ
=== added file 'WebSearch/images/data/Yahoo.png'
Binary files WebSearch/images/data/Yahoo.png 1970-01-01 00:00:00 +0000 and WebSearch/images/data/Yahoo.png 2010-05-22 02:22:22 +0000 differ
=== added directory 'WebSearch/images/twitter'
=== modified file 'WebSearch/lib/Bing.rb'
--- WebSearch/lib/Bing.rb 2010-05-13 01:42:09 +0000
+++ WebSearch/lib/Bing.rb 2010-05-22 02:22:22 +0000
@@ -1,6 +1,7 @@
class Bing < Engine
def initialize
+ self.name = self.class.to_s
self.base_url = "http://www.bing.com";
self.query_url = "#{self.base_url}/search?q=" # 10 results per page
super
=== modified file 'WebSearch/lib/Engine.rb'
--- WebSearch/lib/Engine.rb 2010-05-18 14:29:46 +0000
+++ WebSearch/lib/Engine.rb 2010-05-22 02:22:22 +0000
@@ -1,8 +1,10 @@
class Engine # Factory + Inheritance
require './lib/Link.rb'
+ require './lib/Exceptions.rb'
+ require './lib/Engines.rb'
- attr_accessor :name, :stats, :links, :base_url, :query_url, :engine
+ attr_accessor :name, :stats, :links, :base_url, :query_url
def initialize
self.links =[]
@@ -10,17 +12,13 @@
end
def connect
- WebSearch.log "connecting to #{self.name}"
- self.engine = case self.name
- when "Google"; require './lib/Google.rb'; Google.new # lazy loading applied to engines libraries
- when "Bing"; require './lib/Bing.rb'; Bing.new
- when "Yahoo!"; require './lib/Yahoo.rb'; Yahoo.new
- when "Teoma"; require './lib/Teoma.rb'; Teoma.new
- when "Wikipedia"; require './lib/Wikipedia.rb'; Wikipedia.new
- when "Youtube"; require './lib/Youtube.rb'; Youtube.new
- when "Webshots"; require './lib/Webshots.rb'; Webshots.new
- when "Flickr"; require './lib/Flickr.rb'; Flickr.new
- when "ImageShack"; require './lib/ImageShack.rb'; ImageShack.new
+ WebSearch.log "trying to connect to #{self.name} lib"
+ # lazy loading, e.g, when "Google"; require "./lib/Google.rb"; Google.new
+ # The Yahoo! "!" signal was removed
+ if Engines.exists? self.name
+ require "./lib/#{self.name}.rb"; Kernel.const_get(self.name).new
+ else
+ raise Exceptions::UnknownEngineException.new(self.name)
end
end
=== added file 'WebSearch/lib/Engines.rb'
--- WebSearch/lib/Engines.rb 1970-01-01 00:00:00 +0000
+++ WebSearch/lib/Engines.rb 2010-05-22 02:22:22 +0000
@@ -0,0 +1,22 @@
+module Engines
+
+ GOOGLE = "Google"
+ BING = "Bing"
+ YAHOO = "Yahoo"
+ TEOMA = "Teoma"
+ WIKIPEDIA = "Wikipedia"
+ YOUTUBE = "Youtube"
+ WEBSHOTS = "Webshots"
+ FLICKR = "Flickr"
+ IMAGESHACK = "ImageShack"
+ TWITTER = "Twitter"
+ List = [GOOGLE, BING, YAHOO, TEOMA, WIKIPEDIA, YOUTUBE, WEBSHOTS, FLICKR, IMAGESHACK, TWITTER]
+
+ def self.exists? (engine)
+ List.include? engine
+ end
+
+ def self.at (index)
+ List.at index
+ end
+end
=== modified file 'WebSearch/lib/Flickr.rb'
--- WebSearch/lib/Flickr.rb 2010-05-13 01:42:09 +0000
+++ WebSearch/lib/Flickr.rb 2010-05-22 02:22:22 +0000
@@ -1,6 +1,7 @@
class Flickr < Engine
def initialize
+ self.name = self.class.to_s
self.base_url = "http://www.flickr.com";
self.query_url = "#{self.base_url}/search/?q=" # 28 results per page
super
=== modified file 'WebSearch/lib/Google.rb'
--- WebSearch/lib/Google.rb 2010-05-18 14:29:46 +0000
+++ WebSearch/lib/Google.rb 2010-05-22 02:22:22 +0000
@@ -4,6 +4,7 @@
attr_accessor :number_of_fetched_links
def initialize
+ self.name = self.class.to_s
self.number_of_fetched_links = 100
self.base_url = "http://www.google.com";
self.query_url = "#{self.base_url}/search?q=" # (10,20,30,50,100) results per page"
@@ -32,6 +33,6 @@
stats = (google/"div[@id='resultStats']")
/^About ([\S]+) results \s\(([\S]+) seconds\)/.match(stats.inner_text)
total, time = $1, $2
- "Search for #{query} using #{Engine.name} returned #{total} results in #{time} seconds"
+ "Search for #{query} returned #{total} results in #{time} seconds"
end
end
=== modified file 'WebSearch/lib/ImageShack.rb'
--- WebSearch/lib/ImageShack.rb 2010-05-18 14:29:46 +0000
+++ WebSearch/lib/ImageShack.rb 2010-05-22 02:22:22 +0000
@@ -1,6 +1,7 @@
class ImageShack < Engine
def initialize
+ self.name = self.class.to_s
self.base_url = "http://imageshack.us";
self.query_url = "#{self.base_url}/new_search.php?q=" # 49 results per page
super
=== modified file 'WebSearch/lib/Link.rb'
--- WebSearch/lib/Link.rb 2010-05-18 14:29:46 +0000
+++ WebSearch/lib/Link.rb 2010-05-22 02:22:22 +0000
@@ -4,13 +4,13 @@
def initialize (url = "", description = "", icon = File.expand_path("./icon"))
self.url = url
- self.description = description
+ self.description = shorten description # description shortened by default
self.id = @@next_id += 1
self.icon = icon
self.shortened_url = shorten url
end
- def shorten (string, count = 45) # TODO: count as a parameter in .conf file
+ def shorten (string, count = 75) # TODO: count as a parameter in .conf file
if string.length > count
shortened = string.slice(0 .. count-1)
shortened + "..." if shortened
@@ -38,8 +38,7 @@
def download_thumbnail # remember that is being threaded outside
# download thumb quietly (q), name it (O) '#{image_id}.jpg' and take it to the directory named as engine
- WebSearch.log "wget #{self.thumb_url} -O #{self.thumb_path}"
- IO.popen("wget '#{self.thumb_url}' -O #{self.thumb_path}") do |io| # important enclose url in single quotes cuz there is '&'
+ IO.popen("wget -q '#{self.thumb_url}' -O #{self.thumb_path}") do |io| # important enclose url in single quotes cuz there is '&'
IO.select([io], nil, nil, 0.5) # non-blocking download through the pipe
end
self.downloaded_thumb = true
@@ -49,7 +48,7 @@
# Thumbnail path composed by the search engine and image id
# Extract from the thumb_url the what is the search engine using the the core of the url
def define_thumbnail_path
- directories = %w(youtube webshots flickr imageshack) # search engine name == directory name
+ directories = %w(youtube webshots flickr imageshack twitter) # search engine name == directory name
directory = directories.detect {|d| self.url.include?(d)} # search for engines names in url
"./images/#{directory}/#{self.image_id}.jpg"
end
=== modified file 'WebSearch/lib/Teoma.rb'
--- WebSearch/lib/Teoma.rb 2010-05-13 01:42:09 +0000
+++ WebSearch/lib/Teoma.rb 2010-05-22 02:22:22 +0000
@@ -1,6 +1,7 @@
class Teoma < Engine
def initialize
+ self.name = self.class.to_s
self.base_url = "http://www.teoma.com";
self.query_url = "#{self.base_url}/web?q=" # 10 results per page
super
=== added file 'WebSearch/lib/Twitter.rb'
--- WebSearch/lib/Twitter.rb 1970-01-01 00:00:00 +0000
+++ WebSearch/lib/Twitter.rb 2010-05-22 02:22:22 +0000
@@ -0,0 +1,25 @@
+class Twitter < Engine
+
+ def initialize
+ self.name = self.class.to_s
+ self.query_url = "http://search.twitter.com/search?q="; # 15 results per page
+ super
+ end
+
+ # url, e.g., http://twitter.com/runscored_cin/statuses/14382443834
+ # thumb_url, e.g., http://a1.twimg.com/profile_images/768793556/n12430139_39051956_3639_normal.jpg ; twitter does not change the original pic name
+ # description, e.g, Dusty, I hope you literally kill Miguel Cairo with your words after today. #RedsFAIL
+ def retrieve_links(query, page = 1)
+ twitter = Nokogiri::HTML.parse(open("#{self.query_url}#{query}&page=#{page}"))
+ (twitter/"div[@id='results']/ul").each do |res|
+ (res/"li").each do |raw_result|
+ thumb_url = raw_result.at("div[@class='avatar']/a/img")['src'] # the thumb of the avatar which tweeted
+ description = raw_result.at("div[@class='msg']/span[@class^='msgtxt']").inner_text # the tweet text
+ url = raw_result.at("div[@class='info']/a[@class='lit']")['href'] # url of the tweet
+ self.links << ThumbnailedLink.new(url, description, thumb_url)
+ end
+ end
+ self.links
+ end
+
+end
=== modified file 'WebSearch/lib/Webshots.rb'
--- WebSearch/lib/Webshots.rb 2010-05-13 01:42:09 +0000
+++ WebSearch/lib/Webshots.rb 2010-05-22 02:22:22 +0000
@@ -1,6 +1,7 @@
class Webshots < Engine
def initialize
+ self.name = self.class.to_s
self.base_url = "http://www.webshots.com";
self.query_url = "#{self.base_url}/search?querySource=community&query=" # 36 results per page
super
=== modified file 'WebSearch/lib/Wikipedia.rb'
--- WebSearch/lib/Wikipedia.rb 2010-05-13 01:42:09 +0000
+++ WebSearch/lib/Wikipedia.rb 2010-05-22 02:22:22 +0000
@@ -3,6 +3,7 @@
attr_accessor :number_of_fetched_links
def initialize
+ self.name = self.class.to_s
self.number_of_fetched_links = 100
self.base_url = "http://en.wikipedia.org";
self.query_url = "#{self.base_url}/w/index.php?title=Special:Search&search=" # parameter "limit" results per page
=== modified file 'WebSearch/lib/Yahoo.rb'
--- WebSearch/lib/Yahoo.rb 2010-05-13 01:42:09 +0000
+++ WebSearch/lib/Yahoo.rb 2010-05-22 02:22:22 +0000
@@ -1,6 +1,7 @@
class Yahoo < Engine
def initialize
+ self.name = self.class.to_s
self.base_url = "http://search.yahoo.com";
self.query_url = "#{self.base_url}/search?p=" # 10 results per page
super
=== modified file 'WebSearch/lib/Youtube.rb'
--- WebSearch/lib/Youtube.rb 2010-05-13 01:42:09 +0000
+++ WebSearch/lib/Youtube.rb 2010-05-22 02:22:22 +0000
@@ -1,6 +1,7 @@
class Youtube < Engine
def initialize
+ self.name = self.class.to_s
self.base_url = "http://www.youtube.com";
self.query_url = "#{self.base_url}/results?search_query=" # 20 results per page
super
