This is a small enough loss that it takes quite a while to manifest, especially when testing at a small scale. Initially, Media Proxy would leak 16 bytes on each request. When comparing lilliput to pillow-simd, we found that lilliput performed as well as or better than pillow-simd in the use cases we care about. Finally with this combination we had a service which steadily beat Image Proxy in synthetic benchmarks. We added fasthttp to handle our concurrent HTTP client and server requirements. libjpeg-turbo for JPEG, libpng for PNG) and OpenCV’s fast vectorized resizing code. Lilliput uses the existing and mature C libraries for image compression and decompression (e.g. In particular, we wanted to be able to inspect image headers before deciding whether to start decompressing them, since this would allow us to immediately refuse to resize any that are too large. Lilliput’s OpenCV wrapper does almost everything we want, though we still had to fork OpenCV slightly before we were happy with it. Lilliput has been made with careful consideration toward not creating garbage in Go. We created our own Go image resizer, named Lilliput, which has its own Cgo wrapper on top of OpenCV. We had seen some promise when we benchmarked one Go package that wrapped OpenCV, but that package didn’t support all of the features we wanted. We decided to double down and put together our own image resizing package for Go. Go might be a little faster at handling HTTP, but if it can’t resize images quickly, the extra speedup would be lost to the extra time spent resizing. Most of the work done by the Image Proxy was in the image transcoding and resizing, so this would be a significant bottleneck in Media Proxy if it were slower. While Go is generally a faster language than Python, none of the resizing packages we could find could beat Pillow-simd consistently in performance. We began by benchmarking existing image resizing packages for Go and quickly discovered something disheartening. In addition to serving requests faster, the new service would also get new features, including the ability to get the first frame from. Thankfully, Image Proxy was relatively simple, and comparing results between it and the replacement service would be straightforward. It’s not an easy decision to rewrite a service that’s already working. We likely could have addressed this behavior in Image Proxy, but we had been experimenting with using more Go, and it seemed like a good place to try Go out. Image proxying requests saw a wide variance of response times, with some taking multiple seconds to complete. The biggest problem was the Image Proxy did not have an even workload distribution, which hampered its throughput. Still, as Discord grew, the Image Proxy started to show signs of strain. This combination of cache and proxy was enough to scale our image proxy up well into millions of users. The cache performs request coalescing in order to minimize the number of resize transformations required. A HAProxy layer routes requests based on a URL hash to the Nginx caching layer. On top of this, we setup a caching layer that would try to keep resized images around in memory and respond directly from cache when it could. The Image Proxy would receive a HTTP request containing a URL to fetch, resize, and finally respond with the image. Pillow-simd is wonderfully fast and uses x86 SSE instructions to accelerate resizing where it can. It fetched images from remote URLs and then used the pillow-simd package to do the heavy lifting of image resizing. To handle this job, we created a Python service creatively named Image Proxy. To circumvent these problems, Discord needs an intermediary service to fetch images on behalf of users and then resize them to reduce bandwidth usage. Linking directly to images would leak users’ IP addresses to image hosts, and large images use up lots of bandwidth. While we wish it was as simple as sending them out across the tubes to your friends, delivering these images creates some pretty large technical problems. Despite being a voice and text chat app, Discord sees over a hundred million images passing through its tubes every day.
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |