[GH-ISSUE #80] Fatal error when processing linked url for embed when site uses unverified certificates #13

Closed
opened 2026-09-02 13:08:56 +02:00 by TheLovinator · 1 comment
Owner

Originally created by @thebetauser on GitHub (Mar 29, 2023).
Original GitHub issue: https://github.com/TheLovinator1/discord-twitter-webhooks/issues/80

The application crashes when it tries to access a site without a valid signed ssl certtificate

(Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:992)')))

Full log below (this build was from ~3 days ago)

2023-03-24 at 17:26:23 ERROR Got an error: discord-twitter-webhooks: An unhandled exception was raised while streaming. Shutting down
Exception: HTTPSConnectionPool(host='pages.stern.nyu.edu', port=443): Max retries exceeded with url: /~pschnabl/research/DSS_SVB.pdf (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:992)')))
Traceback (most recent call last):
  File "/opt/pysetup/.venv/lib/python3.11/site-packages/urllib3/connectionpool.py", line 703, in urlopen
    httplib_response = self._make_request(
                       ^^^^^^^^^^^^^^^^^^^
  File "/opt/pysetup/.venv/lib/python3.11/site-packages/urllib3/connectionpool.py", line 386, in _make_request
    self._validate_conn(conn)
  File "/opt/pysetup/.venv/lib/python3.11/site-packages/urllib3/connectionpool.py", line 1042, in _validate_conn
    conn.connect()
  File "/opt/pysetup/.venv/lib/python3.11/site-packages/urllib3/connection.py", line 419, in connect
    self.sock = ssl_wrap_socket(
                ^^^^^^^^^^^^^^^^
  File "/opt/pysetup/.venv/lib/python3.11/site-packages/urllib3/util/ssl_.py", line 449, in ssl_wrap_socket
    ssl_sock = _ssl_wrap_socket_impl(
               ^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/pysetup/.venv/lib/python3.11/site-packages/urllib3/util/ssl_.py", line 493, in _ssl_wrap_socket_impl
    return ssl_context.wrap_socket(sock, server_hostname=server_hostname)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/ssl.py", line 517, in wrap_socket
    return self.sslsocket_class._create(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/ssl.py", line 1075, in _create
    self.do_handshake()
  File "/usr/local/lib/python3.11/ssl.py", line 1346, in do_handshake
    self._sslobj.do_handshake()
ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:992)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/opt/pysetup/.venv/lib/python3.11/site-packages/requests/adapters.py", line 489, in send
    resp = conn.urlopen(
           ^^^^^^^^^^^^^
  File "/opt/pysetup/.venv/lib/python3.11/site-packages/urllib3/connectionpool.py", line 787, in urlopen
    retries = retries.increment(
              ^^^^^^^^^^^^^^^^^^
  File "/opt/pysetup/.venv/lib/python3.11/site-packages/urllib3/util/retry.py", line 592, in increment
    raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='pages.stern.nyu.edu', port=443): Max retries exceeded with url: /~pschnabl/research/DSS_SVB.pdf (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:992)')))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/opt/pysetup/.venv/lib/python3.11/site-packages/tweepy/asynchronous/streaming.py", line 84, in _connect
    await self.on_data(line)
  File "/opt/pysetup/.venv/lib/python3.11/site-packages/tweepy/asynchronous/streaming.py", line 842, in on_data
    await self.on_response(
  File "/discord_twitter_webhooks/main.py", line 161, in on_response
    main(response)
  File "/discord_twitter_webhooks/main.py", line 73, in main
    twitter_card_image = get.meta_image(entities)
                         ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/discord_twitter_webhooks/get.py", line 66, in meta_image
    response: requests.Response = requests.get(url_list[0], timeout=5)
                                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/pysetup/.venv/lib/python3.11/site-packages/requests/api.py", line 73, in get
    return request("get", url, params=params, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/pysetup/.venv/lib/python3.11/site-packages/requests/api.py", line 59, in request
    return session.request(method=method, url=url, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/pysetup/.venv/lib/python3.11/site-packages/requests/sessions.py", line 587, in request
    resp = self.send(prep, **send_kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/pysetup/.venv/lib/python3.11/site-packages/requests/sessions.py", line 701, in send
    r = adapter.send(request, **kwargs)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/pysetup/.venv/lib/python3.11/site-packages/requests/adapters.py", line 563, in send
    raise SSLError(e, request=request)
requests.exceptions.SSLError: HTTPSConnectionPool(host='pages.stern.nyu.edu', port=443): Max retries exceeded with url: /~pschnabl/research/DSS_SVB.pdf (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:992)')))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/opt/pysetup/.venv/lib/python3.11/site-packages/tweepy/asynchronous/streaming.py", line 135, in _connect
    await self.on_exception(e)
  File "/discord_twitter_webhooks/main.py", line 156, in on_exception
    sys.exit(error_msg)
SystemExit: discord-twitter-webhooks: An unhandled exception was raised while streaming. Shutting down
Exception: HTTPSConnectionPool(host='pages.stern.nyu.edu', port=443): Max retries exceeded with url: /~pschnabl/research/DSS_SVB.pdf (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:992)')))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/discord_twitter_webhooks/main.py", line 225, in <module>
    start()
  File "/discord_twitter_webhooks/main.py", line 221, in start
    asyncio.run(start_bot())
  File "/usr/local/lib/python3.11/asyncio/runners.py", line 190, in run
    return runner.run(main)
           ^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/asyncio/runners.py", line 118, in run
    return self._loop.run_until_complete(task)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/asyncio/base_events.py", line 653, in run_until_complete
    return future.result()
           ^^^^^^^^^^^^^^^
  File "/discord_twitter_webhooks/main.py", line 188, in start_bot
    await stream.filter(
  File "/opt/pysetup/.venv/lib/python3.11/site-packages/tweepy/asynchronous/streaming.py", line 514, in _connect
    await super()._connect(method, url, headers=headers, **kwargs)
  File "/opt/pysetup/.venv/lib/python3.11/site-packages/tweepy/asynchronous/streaming.py", line 137, in _connect
    await self.session.close()

Originally created by @thebetauser on GitHub (Mar 29, 2023). Original GitHub issue: https://github.com/TheLovinator1/discord-twitter-webhooks/issues/80 The application crashes when it tries to access a site without a valid signed ssl certtificate ``` (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:992)'))) ``` Full log below (this build was from ~3 days ago) ``` 2023-03-24 at 17:26:23 ERROR Got an error: discord-twitter-webhooks: An unhandled exception was raised while streaming. Shutting down Exception: HTTPSConnectionPool(host='pages.stern.nyu.edu', port=443): Max retries exceeded with url: /~pschnabl/research/DSS_SVB.pdf (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:992)'))) Traceback (most recent call last): File "/opt/pysetup/.venv/lib/python3.11/site-packages/urllib3/connectionpool.py", line 703, in urlopen httplib_response = self._make_request( ^^^^^^^^^^^^^^^^^^^ File "/opt/pysetup/.venv/lib/python3.11/site-packages/urllib3/connectionpool.py", line 386, in _make_request self._validate_conn(conn) File "/opt/pysetup/.venv/lib/python3.11/site-packages/urllib3/connectionpool.py", line 1042, in _validate_conn conn.connect() File "/opt/pysetup/.venv/lib/python3.11/site-packages/urllib3/connection.py", line 419, in connect self.sock = ssl_wrap_socket( ^^^^^^^^^^^^^^^^ File "/opt/pysetup/.venv/lib/python3.11/site-packages/urllib3/util/ssl_.py", line 449, in ssl_wrap_socket ssl_sock = _ssl_wrap_socket_impl( ^^^^^^^^^^^^^^^^^^^^^^ File "/opt/pysetup/.venv/lib/python3.11/site-packages/urllib3/util/ssl_.py", line 493, in _ssl_wrap_socket_impl return ssl_context.wrap_socket(sock, server_hostname=server_hostname) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/ssl.py", line 517, in wrap_socket return self.sslsocket_class._create( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/ssl.py", line 1075, in _create self.do_handshake() File "/usr/local/lib/python3.11/ssl.py", line 1346, in do_handshake self._sslobj.do_handshake() ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:992) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/opt/pysetup/.venv/lib/python3.11/site-packages/requests/adapters.py", line 489, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/opt/pysetup/.venv/lib/python3.11/site-packages/urllib3/connectionpool.py", line 787, in urlopen retries = retries.increment( ^^^^^^^^^^^^^^^^^^ File "/opt/pysetup/.venv/lib/python3.11/site-packages/urllib3/util/retry.py", line 592, in increment raise MaxRetryError(_pool, url, error or ResponseError(cause)) urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='pages.stern.nyu.edu', port=443): Max retries exceeded with url: /~pschnabl/research/DSS_SVB.pdf (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:992)'))) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/opt/pysetup/.venv/lib/python3.11/site-packages/tweepy/asynchronous/streaming.py", line 84, in _connect await self.on_data(line) File "/opt/pysetup/.venv/lib/python3.11/site-packages/tweepy/asynchronous/streaming.py", line 842, in on_data await self.on_response( File "/discord_twitter_webhooks/main.py", line 161, in on_response main(response) File "/discord_twitter_webhooks/main.py", line 73, in main twitter_card_image = get.meta_image(entities) ^^^^^^^^^^^^^^^^^^^^^^^^ File "/discord_twitter_webhooks/get.py", line 66, in meta_image response: requests.Response = requests.get(url_list[0], timeout=5) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/pysetup/.venv/lib/python3.11/site-packages/requests/api.py", line 73, in get return request("get", url, params=params, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/pysetup/.venv/lib/python3.11/site-packages/requests/api.py", line 59, in request return session.request(method=method, url=url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/pysetup/.venv/lib/python3.11/site-packages/requests/sessions.py", line 587, in request resp = self.send(prep, **send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/pysetup/.venv/lib/python3.11/site-packages/requests/sessions.py", line 701, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/pysetup/.venv/lib/python3.11/site-packages/requests/adapters.py", line 563, in send raise SSLError(e, request=request) requests.exceptions.SSLError: HTTPSConnectionPool(host='pages.stern.nyu.edu', port=443): Max retries exceeded with url: /~pschnabl/research/DSS_SVB.pdf (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:992)'))) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/opt/pysetup/.venv/lib/python3.11/site-packages/tweepy/asynchronous/streaming.py", line 135, in _connect await self.on_exception(e) File "/discord_twitter_webhooks/main.py", line 156, in on_exception sys.exit(error_msg) SystemExit: discord-twitter-webhooks: An unhandled exception was raised while streaming. Shutting down Exception: HTTPSConnectionPool(host='pages.stern.nyu.edu', port=443): Max retries exceeded with url: /~pschnabl/research/DSS_SVB.pdf (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:992)'))) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/discord_twitter_webhooks/main.py", line 225, in <module> start() File "/discord_twitter_webhooks/main.py", line 221, in start asyncio.run(start_bot()) File "/usr/local/lib/python3.11/asyncio/runners.py", line 190, in run return runner.run(main) ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/asyncio/runners.py", line 118, in run return self._loop.run_until_complete(task) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/asyncio/base_events.py", line 653, in run_until_complete return future.result() ^^^^^^^^^^^^^^^ File "/discord_twitter_webhooks/main.py", line 188, in start_bot await stream.filter( File "/opt/pysetup/.venv/lib/python3.11/site-packages/tweepy/asynchronous/streaming.py", line 514, in _connect await super()._connect(method, url, headers=headers, **kwargs) File "/opt/pysetup/.venv/lib/python3.11/site-packages/tweepy/asynchronous/streaming.py", line 137, in _connect await self.session.close() ```
Author
Owner

@TheLovinator1 commented on GitHub (Mar 29, 2023):

Thanks mate!

This is now fixed in 6ff02c53e1.

<!-- gh-comment-id:1487898991 --> @TheLovinator1 commented on GitHub (Mar 29, 2023): Thanks mate! This is now fixed in 6ff02c53e12ea49ba9e34d611ceb0127708fb7fd.
Sign in to join this conversation.
No labels
pull-request
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
TheLovinator/discord-twitter-webhooks#13
No description provided.