"[Errno 98] Address Already in Use" When Using Spotify API with django-crontab

[ahmaraa]

I am working on a Django project where I use a cron job to manage Spotify playlists using the Spotify API. The cron job is scheduled with `django-crontab` and is **intended to add tracks** to multiple Spotify playlists across different accounts.

**Problem:**
The cron job works fine up to a certain point, but I encounter the following error repeatedly:

```
ERROR 2024-09-06 01:11:02,768 cron Error adding tracks: [Errno 98] Address already in use
ERROR 2024-09-06 01:11:07,771 cron Error adding tracks: [Errno 98] Address already in use
ERROR 2024-09-06 01:11:12,774 cron Error adding tracks: [Errno 98] Address already in use
```

**What I have tried:**

- **Environment:** I am using Ubuntu with Django, django-crontab, and Spotipy (a Python client for the Spotify Web API).
- Spotify Web API Sandbox: I tested the API with [spotify web API](https://developer.spotify.com/documentation/web-api/reference/add-tracks-to-playlist) and it works fine there.
- **Error Analysis:** The error occurs after a log statement indicating that a user ID is being used to add tracks to a playlist. The next try block does not execute, and the [Errno 98] Address already in use error is raised.
- **Port Conflict Check:** I initially thought the error might be due to port conflicts with the Django server running on port 8000, but I understand that my Django server is expected to run on this port, so checking for its usage is not necessary.
- **Error Handling and Retrying:** I’ve implemented retry logic to handle exceptions such as Spotify API rate limits (429 Too Many Requests) and permission errors (403 Forbidden). The retry logic handles these exceptions correctly, but the [Errno 98] Address already in use error persists.

```
def add_tracks_to_playlist(order, playlist_id, track_uris, sp, spotify_account):
retries = 3
delay = 5
USER_ID = spotify_account.spotify_key.username

logger.info(f"Using user ID {USER_ID} to add tracks to playlist {playlist_id}")

for attempt in range(retries):
try:
logger.debug(f"Attempting to add {len(track_uris)} tracks to playlist {playlist_id}, attempt {attempt + 1}")
sp.user_playlist_add_tracks(USER_ID, playlist_id, track_uris)
logger.info(f"Successfully added {len(track_uris)} tracks to playlist {playlist_id} for user {order.user_id}")

update_order_status(order)
break # Exit the loop if successful

except SpotifyException as e:
logger.error(f"Spotify API error: {e.msg}")
logger.debug(f"Spotify API response: {e}")

if e.http_status == 403:
logger.error(f"Check if the user {USER_ID} is correctly registered and has permissions to modify the playlist.")
return # Exit further retries as this is likely a permissions issue

if e.http_status == 429: # Rate limit exceeded
retry_after = int(e.headers.get('Retry-After', delay))
logger.info(f"Rate limit exceeded. Retrying after {retry_after} seconds.")
time.sleep(retry_after)

except OSError as e:
if e.errno == 98: # Address already in use
logger.error(f"Error adding tracks: Address already in use. Retrying after {delay} seconds.")
time.sleep(delay)
else:
logger.error(f"OS Error: {e}")
logger.debug(f"Detailed OS exception: {repr(e)}")
time.sleep(delay)

except Exception as e:
logger.error(f"Error adding tracks: {e}")
logger.debug(f"Detailed exception: {repr(e)}")
time.sleep(delay)

else:
logger.error(f"Failed to add tracks to playlist {playlist_id} after {retries} attempts.")

```

**Cronjob log**

```
INFO 2024-09-06 15:29:01,639 cron Cron started
INFO 2024-09-06 15:29:01,640 cron Cron job started.
INFO 2024-09-06 15:29:01,644 cron 1 pending orders found.
INFO 2024-09-06 15:29:01,646 cron Processing Order ID 35 with 1 playlists.
INFO 2024-09-06 15:29:01,653 cron Initializing Spotify client for user myuserid with Client ID: myclientid
INFO 2024-09-06 15:29:01,654 cron Successfully initialized SpotifyOAuth for user.
INFO 2024-09-06 15:29:01,654 cron Assigned jobs: [1]
INFO 2024-09-06 15:29:01,655 cron Distributing 1 playlists to job scheduled at 2024-09-06 15:38:01.655010 for Order ID 35.
INFO 2024-09-06 15:29:01,655 cron Adding track 4jYwFHigUt3c21ATMklTku to playlist 4ZwEDixe0VTBSCHfLLqKoH
ERROR 2024-09-06 15:29:01,659 cron Error adding tracks: [Errno 98] Address already in use
ERROR 2024-09-06 15:29:06,662 cron Error adding tracks: [Errno 98] Address already in use
ERROR 2024-09-06 15:29:11,668 cron Error adding tracks: [Errno 98] Address already in use
```

According to the log cronjob works fine till this line of code `logger.info(f"Using user ID {USER_ID} to add tracks to playlist {playlist_id}")` but when it tries to add song into spotify playlist using spotipy function `sp.user_playlist_add_tracks(USER_ID, playlist_id, track_uris)` it gives me `OS Error 98`

**Additional Information:**

`sudo lsof -i -n -P | grep LISTEN`
```
systemd-r 605 systemd-resolve 14u IPv4 18718 0t0 TCP 127.0.0.53:53 (LISTEN)
cupsd 916 root 6u IPv6 21045 0t0 TCP [::1]:631 (LISTEN)
cupsd 916 root 7u IPv4 21046 0t0 TCP 127.0.0.1:631 (LISTEN)
postman-a 13327 a-h-m-a-r 71u IPv4 111201 0t0 TCP 127.0.0.1:10533 (LISTEN)
python 25924 a-h-m-a-r 8u IPv4 224162 0t0 TCP 127.0.0.1:8000 (LISTEN)
```

- The error occurs intermittently and seems to be related to network or socket handling.
- The cron job is designed to run at specific intervals, and I have a cache-based lock to ensure only one instance runs at a time.

**My Setup:**
- Django Version: latest
- Python Version: 3.12
- Spotipy for adding songs to playlist
- OS: Ubuntu 20.04
- Tools: `django-crontab` for scheduling the cron job

**Question:**
- What could be causing the `[Errno 98] Address already in use` error within the context of my cron job?
- How can I modify my code or cron job setup to avoid this error while ensuring reliable execution of my task?
- Are there best practices or alternative approaches for handling such network-related errors in a cron job environment in Django?

**Wget Request Sample:**
```
wget --quiet \
--method POST \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer 1POdFZRZbvb...qqillRxMr2z' \
--body-data '{\n "uris": [\n "3yHyiUDJdz02FZ6jfUbsmY"\n ],\n "position": 0\n}' \
--output-document \
- 'https://api.spotify.com/v1/playlists/4ZwEDixe0VTBSCHfLLqKoH/tracks?uris=spotify%3Atrack%3A3yHyiUDJdz...'
```

[LambertSpot]

Address already in use sounds like an issue with sockets/ports. Maybe you are trying to spin up another instance on the same one that is already in use? Or the old one has not had time to wind down and release the port? Anyway, not related to the API itself. Good luck with the troubleshooting! Maybe someone else in the community has some helpful guidance.