Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

could not accept new connection #38

Open
vsystech opened this issue Dec 9, 2017 · 10 comments
Open

could not accept new connection #38

vsystech opened this issue Dec 9, 2017 · 10 comments

Comments

@vsystech
Copy link

vsystech commented Dec 9, 2017

Dear Developer, thank your job, but i have a problem with your script.

could not accept new connection error.

http://prntscr.com/hl8182

@vsystech
Copy link
Author

vsystech commented Dec 9, 2017

I can't use the proxy with more than ~450 users "clients": 442} not goes upper.

have a few problem.
http://prntscr.com/hla2kl

2017-12-09 19:24:56+0100 [ProxyClient,client] Server disconnecting from client
2017-12-09 19:24:56+0100 [-] Stopping factory <main.ProxyClientFactory instance at 0x7feb8c652638>
2017-12-09 19:24:56+0100 [Uninitialized] Server connection failed ([Failure instance: Traceback (failure with no frames): <class 'twisted.internet.error.ConnectionRefusedError'>: Connection was refused by other side: 111: Connection refused.
])

http://prntscr.com/hla3jo
2017-12-09 19:25:18+0100 [-] Client disconnected (False, 1006, connection was closed uncleanly (WebSocket closing handshake timeout (peer did not finish the opening handshake in time)))

@vsystech
Copy link
Author

vsystech commented Dec 9, 2017

, "clients": 446}

root@Debian-92-stretch-64-minimal ~ # netstat -anp |grep 8892 |wc -l
696

but i currently 3k visitor, so don't understand why not good.

@vsystech
Copy link
Author

dear developer, i'm try with non ssl version. and some error. after 1500-2k connection. not accept new connection.

@ghost
Copy link

ghost commented Dec 14, 2017

Similar issue:

2017-12-14 11:19:18-0500 [HTTPChannel,0,*******.176] WebSocket is open
2017-12-14 11:19:18-0500 [HTTPChannel,0,*******.176] Starting factory <__main__.ProxyClientFactory instance at 0x11cf998>
2017-12-14 11:19:18-0500 [Uninitialized] Server connection failed ([Failure instance: Traceback (failure with no frames): <class 'twisted.internet.error.ConnectionRefusedError'>: Connection was refused by other side: 111: Connection refused.
        ])
2017-12-14 11:19:18-0500 [Uninitialized] Server disconnecting from client

@vphelipe
Copy link

Have you tried changing the ulimit?

@vsystech
Copy link
Author

Yes, ulimit -n 16384

but nothing changed.

@vsystech
Copy link
Author

root@Debian-92-stretch-64-minimal ~ # ulimit -a
core file size (blocks, -c) 0
data seg size (kbytes, -d) unlimited
scheduling priority (-e) 0
file size (blocks, -f) unlimited
pending signals (-i) 63753
max locked memory (kbytes, -l) 64
max memory size (kbytes, -m) unlimited
open files (-n) 16384
pipe size (512 bytes, -p) 8
POSIX message queues (bytes, -q) 819200
real-time priority (-r) 0
stack size (kbytes, -s) 8192
cpu time (seconds, -t) unlimited
max user processes (-u) 63753
virtual memory (kbytes, -v) unlimited
file locks (-x) unlimited

@vphelipe
Copy link

I use this code in one cloud server, with peaks of 8 thousand users and functions well, but I made some modifications, my code is this:

import autobahn.twisted.websocket
import autobahn.twisted.resource
import json
import os
import socket
import sys
import twisted.internet.defer
import twisted.internet.protocol
import twisted.internet.reactor
import twisted.protocols.basic
import twisted.web.resource
import twisted.web.server
import twisted.web.static

from twisted.python import log
from twisted.internet import ssl

def toJson(obj):
    return json.dumps(obj).encode("utf-8")

class Container:

    def __init__(self):
      self.rpcId = 0
      self.workerId = None
      self.hashes = 0
      self.to_client = twisted.internet.defer.DeferredQueue()
      self.to_server = twisted.internet.defer.DeferredQueue()

    def getNextRpcId(self):
        self.rpcId += 1
        return self.rpcId

    #def incAndGetHashes(self):
        #self.hashes += 256
        #return self.hashes

class Root(twisted.web.static.File):
    def directoryListing(self):
        return twisted.web.resource.ForbiddenResource()

class ProxyClient(twisted.protocols.basic.LineOnlyReceiver):

    delimiter = b'\n'

    def connectionMade(self):
        #log.msg('Server connected')
        self.factory.di.to_server.get().addCallback(self.dataEnqueued)
        try:
            self.transport.getHandle().setsockopt(socket.SOL_SOCKET, socket.SO_KEEPALIVE, 1)
            self.transport.getHandle().setsockopt(socket.SOL_TCP, socket.TCP_KEEPIDLE, 60)
            self.transport.getHandle().setsockopt(socket.SOL_TCP, socket.TCP_KEEPINTVL, 1)
            self.transport.getHandle().setsockopt(socket.SOL_TCP, socket.TCP_KEEPCNT, 5)
        except:
            pass

    def dataEnqueued(self, data):
        if data is None:
            #log.msg('Client disconnecting from server')
            self.transport.loseConnection()
        else:
            #log.msg('Queue -> Server: %s' % str(data))
            self.transport.write(data)
            if not data.endswith(self.delimiter):
                self.transport.write(self.delimiter)
            self.factory.di.to_server.get().addCallback(self.dataEnqueued)

    def lineReceived(self, line):
        #log.msg('Server -> Queue: %s' % line)
        data = json.loads(line)
        if data.get("id") == 1:
          self.factory.di.workerId = data.get("result").get("id")
          self.factory.di.to_client.put(b'{"type":"authed","params":{"token":"","hashes":0}}')
          if data.get('result', {}).get('job'):
            self.factory.di.to_client.put(toJson({'type':'job','params':data['result']['job']}))
        elif data.get('method') == 'job':
          self.factory.di.to_client.put(toJson({'type':'job','params':data['params']}))
        #elif data.get('result', {}).get('status') == 'OK':
          #hashes = self.factory.di.incAndGetHashes()
          #self.factory.di.to_client.put(toJson({'type':'hash_accepted','params':{'hashes':hashes}}))

    def connectionLost(self, why):
        #log.msg('Server disconnected (%s)' % str(why))
        self.factory.di.to_client.put(None)

class ProxyClientFactory(twisted.internet.protocol.ClientFactory):
    protocol = ProxyClient

    def __init__(self, container):
        self.di = container

    def clientConnectionFailed(self, connector, why):
        #log.msg('Server connection failed (%s)' % str(why))
        self.di.to_client.put(None)

class ProxyServer(autobahn.twisted.websocket.WebSocketServerProtocol):

    #def onConnect(self, request):
        #log.msg('Client connected (%s)' % str(request))

    def onOpen(self):
        #log.msg('WebSocket is open')
        self.di = Container()
        self.di.to_client.get().addCallback(self.onQueue)
        factory = ProxyClientFactory(self.di)
        twisted.internet.reactor.connectTCP(self.targetHost, self.targetPort, factory)

    def onQueue(self, data):
        if data is None:
            #log.msg('Server disconnecting from client')
            self.sendClose()
        else:
            #log.msg('Queue -> Client: %s' % str(data))
            self.sendMessage(data, False)
            self.di.to_client.get().addCallback(self.onQueue)

    def onMessage(self, data, isBinary):
        #log.msg('Client -> Queue (%s): %s' % ('binary' if isBinary else 'text', str(data)))
        data = json.loads(data)
        if data.get('type') == 'auth':
            login = 'YOUR-WALLET'
            self.di.to_server.put(toJson({'method':'login','params':{'login':login,'pass':self.authPass},'id':self.di.getNextRpcId()}))
        if data.get('type') == 'submit':
            data['params']['id'] = self.di.workerId
            self.di.to_server.put(toJson({'method':'submit','params':data['params'],'id':self.di.getNextRpcId()}))

    def onClose(self, wasClean, code, reason):
        #log.msg('Client disconnected (%s, %s, %s)' % (str(wasClean), str(code), str(reason)))
        self.di.to_server.put(None)

if __name__ == "__main__":
    if len(sys.argv) < 3:
        sys.exit('Usage: python %s <stratum tcp host> <stratum tcp port> [stratum auth password]' % sys.argv[0])

    ws = autobahn.twisted.websocket.WebSocketServerFactory()
    ProxyServer.targetHost = sys.argv[1]
    ProxyServer.targetPort = int(sys.argv[2])
    ProxyServer.authPass = sys.argv[3] if len(sys.argv) > 3 else 'x'
    ws.protocol = ProxyServer

    root = Root('./static')
    root.putChild(b"proxy", autobahn.twisted.resource.WebSocketResource(ws))
    site = twisted.web.server.Site(root)
    twisted.internet.reactor.listenSSL(80,site,ssl.DefaultOpenSSLContextFactory('ctr_key/x.key', 'ctr_key/x.crt'),interface='YOUR-IP-SERVER')
    twisted.internet.reactor.run()

Try this code by placing your wallet and your ip from your server. Tell me if you've improved.

@ecSpl01t
Copy link

I have the same problem when I use systemctl for running.

@sasik-github
Copy link

I had same problem There is how I solve for my amount of clients(17K clients):
first I change open files limit
ulimit -n 65536
NOTE: this command should run by user that run you server.
Next increase values for file descriptors
#/etc/security/limits.conf
* - nofile 1048576

next if you have iptables/netfilter(maybe it is not important) increase that value, in my system I did not have this property
#/etc/sysctl.conf
net.ipv4.netfilter.ip_conntrack_max = 1048576

After all this modification my clients online was increased from 300 to 17k and now I meet another problem my python thread with server loaded at 100%. If anyone have any suggestion let me know please.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

5 participants