I ran a redis container with the following command:
docker run --name redis_env --hostname redis \
-p 6379:6379 \
-v $PWD/DBVOL/redis/data:/data:rw \
--privileged=true \
-d redis redis-server
I successfully used Redis Desktop Manager to link this Redis server on this machine, but when I used scrapy-redis to build a distributed crawler according to the document, I got an error message rejecting the connection:
My settings.py is configured as:
# Redis Server settings
REDIS_URL = 'redis://127.0.0.1:6379/0'
# Enable Redis to Schedule Storage Request Queue
SCHEDULER = 'scrapy_redis.scheduler.Scheduler'
# Make sure all crawlers are rewired by redis
DUPEFILTER_CLASS = 'scrapy_redis.dupefilter.RFPDupeFilter'
# Do not clear the Redis queue, so crawling can be paused/resumed
SCHEDULER_PERSIST = True
# Configure item pipelines
# See https://doc.scrapy.org/en/latest/topics/item-pipeline.html
ITEM_PIPELINES = {
#'GeoCrawler.pipelines.GeocrawlerPipeline': 300,
'scrapy_redis.pipelines.RedisPipeline': 300
}
I also tried to link redis server directly using python console, and there was no problem:
>>> import redis
>>> redis.connection("redis://localhost:6379/0")
Connection<host=redis://localhost:6379/0,port=6379,db=0>
May I ask what exactly is the reason? How to solve it? Please give your advice, thank you! ! !
Found the problem, is the ip problem of the container, the ip of the container is wrong, run:
docker inspect -f {{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' container_name_or_id
Just fill in the link string with the ip of the container. But I still don’t understand, why? My rdm can be successfully linked by filling 127.0.0.1: 6379?