Stuttering and getting out of sync after some period

[xphoniex]

I switched from module-simple-protocol-tcp to module-rtp-send after experiencing some stuttering, but it's not fully gone. Not only do I still get break ups, but I also sometimes get sinusoidal break ups in rtp mode, meaning e.g. on and off every 0.25 sec.

Simple protocol has a consistent lag of 300 ms, which is similar to rtp initially but in rtp mode it drifts and needs restarts to get it back to 300 ms. Sometimes there seems to be a burst of packets which makes the sync worse in rtp. In simple protocol, it just goes silent and comes back in sync again.

Are these issues caused by my network? or the receiving device? Anything I could do to tune these?

Also would encoding to something like mp3 be beneficial in terms of bandwidth and stuttering? It seems like it's sending raw audio, 230 kb/s.