Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Using native Redis commandsfor DB Size #17

Open
wants to merge 2 commits into
base: master
Choose a base branch
from
Open

Using native Redis commandsfor DB Size #17

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
02c2d70
Using native Redis commandsfor DB Size
abramovic Sep 24, 2013
0671fc9
Adding Version and new Require Logic
abramovic Sep 24, 2013
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
27 changes: 14 additions & 13 deletions cloud-crawler/lib/cloud-crawler.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,17 +20,18 @@
#
require 'rubygems'
require 'bundler/setup'
require 'cloud-crawler'
require 'cloud-crawler/logger'
require 'cloud-crawler/cookie_store'
require 'cloud-crawler/http'
require 'cloud-crawler/page'
require 'cloud-crawler/redis_page_store'
require 'cloud-crawler/driver'
require 'cloud-crawler/dsl_core'
require 'cloud-crawler/dsl_front_end'
require 'cloud-crawler/crawl_job'
require 'cloud-crawler/batch_crawl_job'
require 'cloud-crawler/batch_curl_job'
require 'cloud-crawler/worker'

%w[logger
cookie_store
http
page
redis_page_store
driver
dsl_core
dsl_front_end
crawl_job
batch_crawl_job
batch_curl_job
worker].each do |f|
require "cloud-crawler/#{f}"
end
2 changes: 1 addition & 1 deletion cloud-crawler/lib/cloud-crawler/redis_page_store.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -97,7 +97,7 @@ def merge!(hash)
end

def size
@pages.keys("*").size
@pages.dbsize
end

def keys
Expand Down
27 changes: 27 additions & 0 deletions cloud-crawler/lib/cloud-crawler/version.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
#
# Copyright (c) 2013 Charles H Martin, PhD
#
# Calculated Content (TM)
# http://calculatedcontent.com
# [email protected]
#
# All rights reserved.
#
# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
# ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
# WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
# DISCLAIMED. IN NO EVENT SHALL MADE BY MADE LTD BE LIABLE FOR ANY
# DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
# (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
# LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
# ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
# SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
#

module CloudCrawler
MAJOR = 0
MINOR = 2
PATCH = 0
VERSION = [MAJOR, MINOR, PATCH].join('.')
end