Whenever we turn on Jenkins on our build server at work, every day in the morning (when people arrive) HTTP traffic on our internet connection seems to be broken.

It only happens when Jenkins is active and our Jenkins jobs are running. The amount of jobs doesn't seem to influence the occurance of the issue.
When I let Jenkins running but disable all jobs, the problem goes away.
It happens also with a newer Jenkins version(1.509.1).

When our HTTP traffic is broken, data in- or out-bound on port 80 and 443 fails most of the time (but not always). Other data comes through just fine (e.g. email, SSH).

We ran a software package on the server to see the active connections on that machine, but we couldn't see anything out of the ordinary.

Whenever our internet is broken, the only way to fix it is to restart our modem. We tried installing a newer Fritz!Box, but it didn't solve the problem.

I have Googled this issue, but I didn't get any closer to finding a possible cause (let alone a solution) to this problem.

Situation:
We have a LAN where our build machine is placed in. This build machine is only accessible from the LAN itself. The LAN is connected to the internet through a switch and then to a Fritz!Box modem/router.
Other devices connect to the LAN (and the build machine) through cable or WiFi.

[edit] Could it be that Jenkins or one of the plugins opens tons of connections and that these connections are never closed?
E.g. could it be the git plugin that polls every minute for new source code and that it doesn't close the connection properly after each poll?
[edit2] Our modem logs also didn't show anything suspicious