Skip to content
GitLab
Explore
Sign in
Primary navigation
Search or go to…
Project
S
ScrapyTutorial
Manage
Activity
Members
Labels
Plan
Issues
Issue boards
Milestones
Wiki
Code
Merge requests
Repository
Branches
Commits
Tags
Repository graph
Compare revisions
Snippets
Build
Pipelines
Jobs
Pipeline schedules
Artifacts
Deploy
Releases
Container Registry
Model registry
Operate
Environments
Monitor
Incidents
Analyze
Value stream analytics
Contributor analytics
CI/CD analytics
Repository analytics
Model experiments
Help
Help
Support
GitLab documentation
Compare GitLab plans
Community forum
Contribute to GitLab
Provide feedback
Keyboard shortcuts
?
Snippets
Groups
Projects
Show more breadcrumbs
Anton Gusev
ScrapyTutorial
Commits
8a0f219c
Commit
8a0f219c
authored
5 years ago
by
Anton Gusev
Browse files
Options
Downloads
Patches
Plain Diff
add dump, statistics
parent
af956b48
No related branches found
No related tags found
5 merge requests
!5
Develop
,
!4
Develop
,
!3
Develop
,
!2
Develop
,
!1
Develop
Changes
4
Hide whitespace changes
Inline
Side-by-side
Showing
4 changed files
.gitignore
+1
-0
1 addition, 0 deletions
.gitignore
image/.gitignore
+0
-1
0 additions, 1 deletion
image/.gitignore
sql/create-dump.sql
+55
-0
55 additions, 0 deletions
sql/create-dump.sql
statistics.txt
+16
-14
16 additions, 14 deletions
statistics.txt
with
72 additions
and
15 deletions
.gitignore
+
1
−
0
View file @
8a0f219c
.*
image
!.gitignore
!.env.example
alembic.ini
...
...
This diff is collapsed.
Click to expand it.
image/.gitignore
deleted
100644 → 0
+
0
−
1
View file @
af956b48
*
\ No newline at end of file
This diff is collapsed.
Click to expand it.
sql/create-dump.sql
0 → 100644
+
55
−
0
View file @
8a0f219c
-- --------------------------------------------------------
-- Хост: 127.0.0.1
-- Версия сервера: 8.0.15 - MySQL Community Server - GPL
-- Операционная система: Win64
-- HeidiSQL Версия: 10.1.0.5464
-- --------------------------------------------------------
/*!40101 SET @OLD_CHARACTER_SET_CLIENT=@@CHARACTER_SET_CLIENT */
;
/*!40101 SET NAMES utf8 */
;
/*!50503 SET NAMES utf8mb4 */
;
/*!40014 SET @OLD_FOREIGN_KEY_CHECKS=@@FOREIGN_KEY_CHECKS, FOREIGN_KEY_CHECKS=0 */
;
/*!40101 SET @OLD_SQL_MODE=@@SQL_MODE, SQL_MODE='NO_AUTO_VALUE_ON_ZERO' */
;
-- Дамп структуры для таблица db-scrapy-tutorial.alembic_version
CREATE
TABLE
IF
NOT
EXISTS
`alembic_version`
(
`version_num`
varchar
(
32
)
NOT
NULL
,
PRIMARY
KEY
(
`version_num`
)
)
ENGINE
=
InnoDB
DEFAULT
CHARSET
=
utf8
;
-- Экспортируемые данные не выделены.
-- Дамп структуры для таблица db-scrapy-tutorial.books
CREATE
TABLE
IF
NOT
EXISTS
`books`
(
`id`
int
(
11
)
NOT
NULL
AUTO_INCREMENT
,
`url`
varchar
(
255
)
DEFAULT
NULL
,
`title`
varchar
(
255
)
NOT
NULL
,
`description`
text
,
`image_path`
varchar
(
255
)
DEFAULT
NULL
,
`rating`
smallint
(
6
)
DEFAULT
NULL
,
`upc`
varchar
(
32
)
DEFAULT
NULL
,
`product_type`
varchar
(
32
)
DEFAULT
NULL
,
`price_excl_tax`
decimal
(
6
,
2
)
DEFAULT
NULL
,
`price_incl_tax`
decimal
(
6
,
2
)
DEFAULT
NULL
,
`tax`
decimal
(
6
,
2
)
DEFAULT
NULL
,
`in_stock`
int
(
11
)
DEFAULT
NULL
,
`count_reviews`
int
(
11
)
DEFAULT
NULL
,
`category`
varchar
(
32
)
DEFAULT
NULL
,
`currency_type`
varchar
(
4
)
DEFAULT
NULL
,
PRIMARY
KEY
(
`id`
),
UNIQUE
KEY
`upc`
(
`upc`
),
KEY
`ix_books_category`
(
`category`
),
KEY
`ix_books_count_reviews`
(
`count_reviews`
),
KEY
`ix_books_currency_type`
(
`currency_type`
),
KEY
`ix_books_in_stock`
(
`in_stock`
),
KEY
`ix_books_price_excl_tax`
(
`price_excl_tax`
),
KEY
`ix_books_price_incl_tax`
(
`price_incl_tax`
),
KEY
`ix_books_product_type`
(
`product_type`
),
KEY
`ix_books_rating`
(
`rating`
),
KEY
`ix_books_tax`
(
`tax`
),
KEY
`ix_books_title`
(
`title`
)
)
ENGINE
=
InnoDB
AUTO_INCREMENT
=
2001
DEFAULT
CHARSET
=
utf8
;
-- Экспортируемые данные не выделены.
/*!40101 SET SQL_MODE=IFNULL(@OLD_SQL_MODE, '') */
;
/*!40014 SET FOREIGN_KEY_CHECKS=IF(@OLD_FOREIGN_KEY_CHECKS IS NULL, 1, @OLD_FOREIGN_KEY_CHECKS) */
;
/*!40101 SET CHARACTER_SET_CLIENT=@OLD_CHARACTER_SET_CLIENT */
;
This diff is collapsed.
Click to expand it.
statistics.txt
+
16
−
14
View file @
8a0f219c
log_count/INFO: 10
start_time: 2019-06-13 14:08:50.725327
scheduler/enqueued/memory: 1
scheduler/enqueued: 1
scheduler/dequeued/memory: 1
scheduler/dequeued: 1
downloader/request_count: 2
downloader/request_method_count/GET: 2
downloader/request_bytes: 444
start_time: 2019-06-14 07:02:37.424266
scheduler/enqueued/memory: 1050
scheduler/enqueued: 1050
scheduler/dequeued/memory: 1050
scheduler/dequeued: 1050
downloader/request_count: 2051
downloader/request_method_count/GET: 2051
downloader/request_bytes: 623523
robotstxt/request_count: 1
downloader/response_count: 2
downloader/response_count: 2
051
downloader/response_status_count/404: 1
downloader/response_bytes: 6204
log_count/DEBUG: 2
response_received_count: 2
downloader/response_bytes: 46033776
response_received_count: 2051
robotstxt/response_count: 1
robotstxt/response_status_count/404: 1
downloader/response_status_count/200: 1
downloader/response_status_count/200: 2050
request_depth_max: 50
file_count: 1000
file_status_count/downloaded: 1000
item_scraped_count: 1000
This diff is collapsed.
Click to expand it.
Preview
0%
Loading
Try again
or
attach a new file
.
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Save comment
Cancel
Please
register
or
sign in
to comment