DEV Community: Anirban Roy

Why I Chose P2P Instead of S3 for My File Transfer Product

Anirban Roy — Wed, 23 Sep 2026 04:34:59 +0000

When I started thinking seriously about building a large-file transfer product, Amazon S3 felt like the obvious architecture.

It is reliable.

It scales.

It has excellent tooling.

Every backend framework knows how to work with it.

And the standard implementation is incredibly straightforward:

User selects file
      ↓
Upload to S3
      ↓
Generate download URL
      ↓
Recipient downloads file

For most products, that is a perfectly reasonable design.

In fact, if I were building document storage, backups, team collaboration, or a Dropbox-style product, I would probably reach for object storage immediately.

But ButterShare was supposed to solve a slightly different problem.

I was not trying to build:

a place where users store files

I was trying to build:

a way to move a file from one device to another

That distinction eventually changed the entire architecture.

Instead of making S3 the center of the product, I started experimenting with direct peer-to-peer transfer using WebRTC.

The architecture became closer to:

Sender
   ↓
Receiver

instead of:

Sender
   ↓
S3
   ↓
Receiver

That sounds like a small change.

It is not.

It changes bandwidth economics, infrastructure complexity, user experience, privacy assumptions, failure modes, and even what the product fundamentally is.

This is why I chose P2P instead of building ButterShare around S3.

First: S3 is not the problem

I want to make this clear before going any further.

S3 is excellent.

Object storage is one of the most useful infrastructure primitives we have.

If your application needs files to remain available after the uploader disappears, object storage makes enormous sense.

For example:

profile images
backups
user-generated content
product assets
documents
media libraries
asynchronous downloads
team collaboration
long-term storage

S3 is extremely good at those things.

My decision was not:

S3 is bad.

It was:

S3 solves a different problem than the one I want ButterShare to solve.

That difference is important.

Transfer and storage are not the same thing

Suppose Alice has a 100GB video project.

Bob needs that project.

There are two ways to think about the problem.

Storage-first approach

Alice uploads the file somewhere.

Alice
  ↓
Cloud storage

Bob later downloads it.

Cloud storage
  ↓
Bob

The cloud copy acts as the middleman.

This is asynchronous.

Alice can upload the file, close her computer, go offline, and Bob can download it hours later.

That is incredibly useful.

But there is another model.

Transfer-first approach

Alice and Bob are both online.

Alice's goal is simply:

Get these bytes from my machine to Bob's machine.

So instead:

Alice
  ↓
Bob

No permanent intermediate copy is required.

That is the model that interested me.

The 100GB thought experiment

Large files make the architectural difference easier to see.

Imagine someone transfers a 100GB file.

With an S3-style workflow:

Sender → S3
100GB

Then:

S3 → Recipient
100GB

The user thinks they are performing one transfer.

But from the infrastructure perspective, the data has moved through two major legs.

You also temporarily have to store 100GB.

Now imagine 1,000 transfers like that.

You are no longer building a tiny file-sharing app.

You are operating a serious storage and bandwidth system.

That may be exactly what you want.

But I started asking:

Why should my infrastructure handle the entire file twice if the sender and receiver are online at the same time?

That question pushed me toward P2P.

What an S3 architecture would look like

The conventional architecture is actually very attractive.

Your backend generates a presigned upload URL.

The browser uploads directly to object storage.

Something like:

Browser
   |
   | request upload URL
   v
Backend
   |
   | presigned URL
   v
Browser
   |
   | upload
   v
S3

Then the recipient gets a download URL:

Recipient
   |
   | download
   v
S3

This avoids sending the large file through your application server.

That is already much better than:

Browser
   ↓
Node.js server
   ↓
S3

for huge files.

Your Node server does not need to proxy 100GB.

S3 handles the actual upload and download.

This architecture is mature, simple, and reliable.

So why not stop there?

Because ButterShare's goal was not storage-backed sharing.

The cost model changes as files get bigger

For small files, infrastructure cost is often barely noticeable.

A user sending:

2MB

is not going to destroy your startup.

But the economics look different when your product specifically encourages:

10GB
50GB
100GB
200GB+

Now you care about:

object storage
upload traffic
download traffic
egress
API requests
multipart upload
abandoned uploads
lifecycle cleanup
replication
logging

And most importantly:

the bigger your product succeeds, the more file data your infrastructure has to handle

That creates a very direct relationship:

more users
=
more storage
+
more bandwidth
+
higher infrastructure cost

P2P changes that relationship.

P2P moves the data plane away from my servers

With a peer-to-peer architecture, my backend still exists.

But it mostly handles coordination.

For example:

Sender
   |
   | create session
   v
Signaling server
   |
   | connection information
   v
Receiver

Once WebRTC establishes the peer connection:

Sender <----------------------> Receiver

The actual file can travel directly between them when network conditions allow.

My backend is responsible for things like:

creating transfer sessions
exchanging signaling data
handling room state
coordinating peers
expiring sessions

It does not need to become the permanent home of every file.

That is a very different infrastructure problem.

Control plane vs data plane

This was a useful way for me to think about the architecture.

Control plane

Small messages.

Things like:

Create room
Join room
Peer connected
SDP offer
SDP answer
ICE candidate
Transfer started
Transfer finished

These messages are tiny.

They are easy and cheap to handle.

Data plane

The actual file:

100GB video

That is the expensive part.

With an S3 architecture, the cloud owns the data plane.

With P2P, I try to keep the data plane between the users whenever possible.

That separation is one of the biggest reasons I liked the architecture.

P2P does not mean "no servers"

This is one of the biggest misconceptions around WebRTC.

People hear:

peer-to-peer

and imagine:

No backend
No infrastructure
No server bills

Unfortunately, physics is not that generous.

You still need infrastructure.

For ButterShare, that can include:

application servers
signaling
session coordination
STUN
TURN
monitoring
analytics
abuse prevention
databases or temporary state

And TURN deserves special attention.

TURN is the part that ruins the perfect diagram

In the ideal case:

Sender <----------------------> Receiver

Great.

But users live behind:

NAT
corporate firewalls
carrier-grade NAT
VPNs
restrictive routers
mobile networks

Sometimes a direct connection cannot be established.

Then WebRTC may use a TURN relay.

The architecture becomes:

Sender
   ↓
TURN server
   ↓
Receiver

Now your infrastructure is handling the bandwidth again.

So P2P does not magically eliminate bandwidth costs.

It changes how often you need to pay them.

That means one of the most important metrics in a P2P file-transfer product is:

What percentage of transfers are direct?

If most transfers are direct, the infrastructure economics can be very attractive.

If most transfers require TURN, your architecture starts looking much more bandwidth-heavy.

But TURN is still not S3

Even when a file goes through TURN, there is an important conceptual difference.

With object storage:

Sender
   ↓
Stored file
   ↓
Recipient

With TURN:

Sender
   ↓
Relay
   ↓
Recipient

TURN relays traffic.

It does not need to create a persistent copy of the full file for later downloading.

That means you still avoid many storage-related concerns.

No persistent file storage changes the operational burden

If I stored arbitrary user files, I would immediately inherit an entire category of problems.

For example:

Storage lifecycle

How long do files stay?

1 hour?
24 hours?
7 days?
30 days?

At some point you need cleanup jobs.

Abandoned uploads

What happens when someone uploads 80GB of a 100GB file and closes the browser?

Do you keep the partial upload?

For how long?

Storage quotas

If the product is free, users will eventually test how free it really is.

You need policies.

Abuse

If your servers permanently store arbitrary files, users can upload things you absolutely do not want to host.

Now you need systems around:

abuse reports
takedowns
malware
illegal content
spam
automated storage abuse

Direct transfer does not eliminate every abuse issue.

But not becoming a long-term file host changes the problem dramatically.

Privacy was another reason

There is also a product philosophy difference.

With traditional cloud transfer, the file usually exists in three places during the workflow:

Sender device
Cloud provider
Recipient device

With direct transfer, the goal is closer to:

Sender device
Recipient device

That is appealing for a product whose main job is:

move this file

rather than:

store this file for me

Again, this does not magically make every P2P implementation private or secure.

The details matter.

You still need secure signaling, strong session identifiers, encrypted transport, good session expiration, and careful metadata handling.

But architecturally, not retaining the actual file reduces how much user content my service needs to hold.

I like that.

Why didn't I just use S3 multipart upload?

That was one of the obvious alternatives.

S3 supports multipart uploads, which are excellent for large files.

Instead of uploading one huge object in one request, you upload parts.

Conceptually:

100GB file

Part 1
Part 2
Part 3
...
Part N

If one part fails, you retry that part instead of restarting the entire upload.

It is a proven solution.

But multipart upload solves:

How do I reliably upload a very large object to cloud storage?

My question was different:

Do I need to upload the object to cloud storage at all?

That is the architectural fork.

The cloud-storage architecture has a major UX advantage

There is one area where S3 wins very clearly:

asynchronous transfer.

Suppose Alice uploads a file at 2 PM.

She closes her laptop.

Bob opens the link at midnight.

With S3:

Works perfectly.

With P2P:

Alice is offline.
No transfer.

That is a major limitation.

With ButterShare's direct-transfer model, both peers need to participate in the live transfer.

That is not a small tradeoff.

It fundamentally changes the use case.

Why I accepted that tradeoff

Because I wanted to optimize for:

"I have a huge file.
You are online.
I need to give it to you now."

Examples:

video editor sending footage
photographer sending RAW files
developer sending a VM image
designer sending a large project
friend sending a huge archive
creator moving footage between devices

For this workflow, I do not necessarily need persistent cloud storage.

I need a reliable pipe.

That is the niche.

This is why I don't think P2P replaces S3

They are complementary.

If I need:

storage
persistence
sharing later
multiple downloads
CDN delivery

I would choose object storage.

If I need:

live transfer
one sender
one receiver
large file
both online

P2P starts looking very attractive.

The architecture should follow the use case, not ideology.

The biggest advantage: no duplicate cloud journey

This is the simplest way I explain the product to myself.

With cloud storage:

Your file is here.

Destination is there.

But first send it somewhere else.

That "somewhere else" is valuable because it gives you persistence.

But if persistence is unnecessary, it can feel redundant.

With P2P:

Source → Destination

That directness becomes more meaningful the larger the file is.

Large files expose the difference

Imagine transferring a 20MB PDF.

Does the architecture matter?

Probably not to the user.

Whether it goes:

Sender → S3 → Receiver

or:

Sender → Receiver

the whole thing may happen quickly.

Now imagine:

150GB

The intermediate step becomes much more visible.

The user may have to upload 150GB before the recipient can even start downloading, depending on the implementation.

Direct transfer lets the destination receive data as it is being sent.

Time-to-first-byte is different too

In a standard upload-then-share flow:

Upload 100GB
      ↓
Finish upload
      ↓
Recipient starts

In a direct-transfer flow:

Sender starts
      ↓
Recipient starts receiving

almost immediately after the connection is established.

This changes the experience.

The receiver does not necessarily have to wait for an entire cloud upload to finish first.

The bandwidth bottleneck still exists

P2P does not make the user's internet connection faster.

This is important.

Suppose:

Sender upload:
20 Mbps

Receiver download:
500 Mbps

The transfer is still constrained by the sender.

A 100GB file is still a 100GB file.

WebRTC does not violate physics.

The architectural advantage is not:

the file becomes tiny

It is:

the sender's upload is going toward the actual destination rather than an intermediate storage system first

That is a much more honest way to think about it.

P2P also creates harder engineering problems

Choosing P2P was not choosing the easier architecture.

In many ways, S3 would have been much easier.

With S3, you get:

durable storage
resumable multipart uploads
globally mature infrastructure
reliable HTTP downloads
straightforward authorization
mature SDKs
monitoring
excellent tooling

With P2P, I have to think about:

signaling
NAT traversal
TURN
WebRTC Data Channels
browser differences
connection failures
memory
backpressure
reconnection
resumability
long-running browser sessions
mobile background behavior

So why do it?

Because those engineering problems are directly aligned with the product I want.

S3 turns the server side into the easy part

Imagine implementing the S3 version.

Your backend might create a presigned URL:

const uploadUrl = await createPresignedUploadUrl({
  key: fileId,
  contentType: file.type
})

The client uploads:

await fetch(uploadUrl, {
  method: "PUT",
  body: file
})

Then you save metadata:

file ID
owner
size
storage key
expiration

Finally, create a download URL.

That is relatively straightforward.

Now compare that to P2P.

The P2P version requires connection negotiation

Before a single byte moves, I need:

Sender joins room
Receiver joins room
Exchange SDP
Exchange ICE candidates
Attempt direct connection
Possibly fallback to TURN
Open Data Channel
Negotiate transfer metadata
Start streaming chunks

That is much more complicated.

The cost benefit comes with engineering complexity.

There is no free lunch.

The first naive P2P implementation was terrible

The first instinct is:

const buffer = await file.arrayBuffer()
channel.send(buffer)

Works great for a small screenshot.

Then someone selects:

50GB

Goodbye browser.

Large P2P transfer requires chunking.

Something like:

const CHUNK_SIZE = 64 * 1024

let offset = 0

while (offset < file.size) {
  const chunk = file.slice(offset, offset + CHUNK_SIZE)
  const buffer = await chunk.arrayBuffer()

  channel.send(buffer)

  offset += chunk.byteLength
}

But even that is not enough.

You also need flow control.

Backpressure became one of the most important concepts

If JavaScript produces chunks faster than the connection sends them, WebRTC queues them.

You can see that queue using:

channel.bufferedAmount

If you ignore it:

send
send
send
send
send
send

eventually memory starts growing.

So a real sender needs to respect the buffer.

For example:

async function waitForDrain(channel) {
  const MAX_BUFFER = 4 * 1024 * 1024

  while (channel.bufferedAmount > MAX_BUFFER) {
    await new Promise(resolve => setTimeout(resolve, 10))
  }
}

Then:

while (offset < file.size) {
  await waitForDrain(channel)

  const chunk = file.slice(offset, offset + CHUNK_SIZE)
  const buffer = await chunk.arrayBuffer()

  channel.send(buffer)

  offset += chunk.byteLength
}

This is the type of problem you never need to think about when S3 is handling everything.

The receiver has an even bigger challenge

Suppose I successfully stream a 100GB file.

What happens on the receiver?

The naive version:

const chunks = []

channel.onmessage = event => {
  chunks.push(event.data)
}

Then:

const blob = new Blob(chunks)

That is fine for a 20MB file.

For 100GB, it is ridiculous.

You need progressive writing.

The receiver needs to write chunks somewhere as they arrive rather than keeping the entire file in memory.

That means browser filesystem APIs, writable streams, or other platform-specific strategies.

This was one of the moments where I realized:

Direct large-file transfer is basically a systems problem disguised as a web app.

Browser compatibility becomes infrastructure

If you build around S3 and HTTP, browsers already know how to upload and download files very well.

When you move more of the transfer engine into the browser, browser behavior becomes part of your infrastructure.

You now care deeply about:

Chrome
Edge
Firefox
Safari
iOS Safari
Android Chrome

And they do not all behave identically.

You care about:

filesystem APIs
memory behavior
background tab suspension
WebRTC quirks
download handling
mobile lifecycle

Your "server infrastructure" may be smaller.

But your client-side complexity becomes much larger.

Mobile is where P2P gets painful

A desktop user might leave a browser tab open for an hour.

A mobile user will:

switch apps
lock their phone
receive a call
enable battery saver
move between Wi-Fi and cellular
let the OS suspend the browser

For a storage-backed upload, the architecture can sometimes recover more naturally.

For a live P2P connection, both devices matter.

That makes mobile reliability one of the hardest parts.

Reconnection is mandatory for huge files

If I am asking someone to transfer:

100GB

I cannot reasonably say:

If your Wi-Fi drops after 99GB, please start again.

That is unacceptable.

P2P means I need application-level resume logic.

For example, the receiver can say:

{
  "fileId": "abc123",
  "receivedBytes": 84639214822
}

The sender can then continue from:

file.slice(receivedBytes)

Of course, a real implementation needs more validation than that.

But resumability becomes part of your transfer protocol.

S3 gives you mature multipart-upload patterns.

With P2P, you own this problem.

So why not let S3 handle failures for me?

Because every architecture has tradeoffs.

S3 gives me easier reliability.

P2P gives me a different economic and product model.

I decided the complexity was worth exploring because the defining promise of ButterShare is not:

We built another upload box.

It is:

Move large files directly between devices.

If I hide an S3 upload behind the UI, I am building a different product.

The economics become especially interesting for free products

This mattered to me because ButterShare is designed to be accessible without charging someone just because their file is large.

Cloud storage products usually have a very understandable reason for limits.

If every free user can upload:

200GB

and leave it sitting on your infrastructure, costs can explode.

So products impose:

storage caps
transfer caps
expiration
paid tiers
monthly limits

That is completely rational.

But if the service does not permanently store the file, the economics change.

It does not make bandwidth free.

It does not eliminate TURN.

But the business is no longer paying to keep every file around.

That opens up different product possibilities.

Storage creates another problem: unused files

Suppose a user uploads 70GB.

They generate a link.

The recipient never downloads it.

With object storage:

you still stored 70GB

Maybe only for a day.

Maybe a week.

But the resource was consumed.

With live P2P transfer:

no receiver
=
no 70GB transfer

This aligns infrastructure usage more closely with actual transfers.

I find that elegant.

Multiple recipients change the answer

Now let's say Alice needs to send the same 100GB file to:

100 people

Suddenly P2P is less obviously attractive.

Alice would potentially need to upload enormous amounts of data repeatedly.

This is where cloud storage/CDN architecture becomes much better.

Upload once:

Alice → Cloud

Then distribute:

Cloud → 100 recipients

That is exactly what cloud infrastructure is good at.

Again:

P2P is not universally better.

It is optimized for a different topology.

My ideal P2P use case

The architecture makes the most sense when the shape looks like this:

1 sender
1 receiver
both online
large file
one-time transfer
no long-term storage required

That is the core scenario I optimize for.

Once the requirements move away from that, object storage becomes increasingly attractive.

S3 would win for collaboration

Imagine building:

shared project folders
version history
team permissions
comments
persistent links
multiple devices

I would absolutely want centralized storage.

Trying to turn pure P2P into a Dropbox replacement would create unnecessary pain.

ButterShare is intentionally focused on transfer.

Not collaboration storage.

That product boundary matters.

Architecture affects the user interface too

With S3, I could show:

Upload complete.

You can close this page.

With P2P, I need to say:

Keep this tab open while the transfer is in progress.

That is objectively less convenient in some situations.

But P2P gives me advantages elsewhere:

no cloud upload stage
no account needed for storage
no storage quota in the traditional sense
no waiting for a full upload before live transfer can proceed
direct cross-device workflow

The UX tradeoff reflects the architecture.

Architecture also changes what "file size limit" means

In a storage-backed system, limits may come from:

plan restrictions
maximum object size
upload gateway limits
account storage quotas
infrastructure policy

With a streaming P2P design, I do not need to allocate storage equal to the entire file on my server.

So the application can avoid imposing a traditional cloud-storage file-size cap.

But practical constraints still exist.

For example:

sender storage
receiver storage
browser capabilities
network stability
available memory
transfer duration
device behavior

So "no file size limit" should never mean:

There are no physical limitations anywhere.

It means the product is not intentionally saying:

Sorry, 5GB maximum.
Pay $20.

just because the service would otherwise have to store the file.

One thing I really like: simpler data retention

If the file is not stored on my infrastructure, there is less file data to manage after the transfer.

I do not need a lifecycle like:

uploaded
available
downloaded
expired
scheduled for deletion
deleted

for every file object.

The transfer session can be much more ephemeral.

Something like:

created
peer joined
connected
transferring
completed
expired

That feels much closer to the product I wanted to build.

But metadata still needs discipline

Not storing file bytes does not mean:

collect everything else forever

You still need to think about what metadata is necessary.

For example:

Do I really need to permanently store:

filename
file size
IP address
recipient metadata
transfer history

for every transfer?

Maybe not.

A privacy-conscious architecture is not just about avoiding object storage.

It is also about minimizing unnecessary metadata.

Security still matters enormously

WebRTC transport is encrypted, but security does not end there.

You still need to think about:

session entropy
room guessing
unauthorized peers
session expiration
signaling abuse
rate limits
replay
malformed messages

For example, if your room ID is too predictable:

ABC001
ABC002
ABC003

then your architecture has a problem even if the underlying transport is encrypted.

The entire system matters.

P2P makes abuse different, not nonexistent

This is another important distinction.

If I am not storing files, I reduce my role as a file host.

But users can still abuse:

signaling endpoints
TURN bandwidth
session creation
automated connections
resource-intensive transfers

So abuse prevention becomes more network-oriented.

You may need:

rate limiting
session limits
TURN credentials
temporary tokens
connection quotas

You have fewer storage-abuse problems but more real-time infrastructure concerns.

TURN abuse can become very expensive

Imagine someone intentionally forces relay connections and pushes huge amounts of traffic through your TURN servers.

Congratulations.

Your supposedly cheap P2P application just became a bandwidth pipe.

TURN credentials should not be treated casually.

Short-lived credentials, authentication, rate limiting, and monitoring matter.

This is one area where P2P architecture can surprise people.

Metrics I care about more than storage usage

In an S3 architecture, I might watch:

storage consumed
object count
download bandwidth
upload bandwidth
lifecycle cleanup

In P2P, I care more about:

peer connection success rate
direct connection percentage
TURN percentage
average transfer size
average transfer duration
completion rate
disconnect rate
resume rate
browser failure rate

These numbers tell me whether the architecture is actually working.

Completion rate matters more than theoretical savings

Let's say P2P reduces infrastructure cost dramatically.

Great.

But if:

30% of 100GB transfers fail

then the architecture is a failure.

Users do not care that my AWS bill is low.

The product has to work.

That is why my priority order is roughly:

1. Reliability
2. Compatibility
3. UX
4. Performance
5. Infrastructure efficiency

Cheap broken software is still broken software.

Why I wouldn't build this with my Node.js server in the middle

There is another architecture I could have chosen:

Sender
   ↓
Node.js server
   ↓
Receiver

No S3.

Just stream everything through my backend.

For small transfers, this can work.

For huge transfers, I would be turning my app server into a giant bandwidth proxy.

Every transfer consumes:

inbound bandwidth
outbound bandwidth
open connections
server resources

At scale, that does not sound attractive.

If my goal is direct transfer, WebRTC is much more aligned with the architecture.

What about WebSockets?

WebSockets are excellent for real-time communication.

I could theoretically do:

Sender
   ↓
WebSocket server
   ↓
Receiver

But again, the server is carrying every byte.

For a chat message:

perfect

For:

100GB

I would rather not make my server the middleman if I can avoid it.

That is another reason WebRTC Data Channels are interesting.

Why WebRTC instead of reinventing transport

Browsers already expose WebRTC.

It gives me:

encrypted transport
peer connection negotiation
ICE
STUN/TURN integration
Data Channels
cross-browser networking primitives

That is a huge amount of functionality.

Building a native P2P protocol across every platform would be much more difficult.

The browser becomes the common runtime.

That means:

Windows
Mac
Linux
Android
iPhone

can potentially participate without dedicated native clients.

That distribution advantage matters a lot to me.

The browser is also the biggest limitation

The beauty is:

No app installation.

The pain is:

You are inside a browser.

You do not control:

lifecycle
memory limits
background execution
filesystem support
OS behavior

So the architecture is a constant tradeoff.

The browser gives incredible reach.

Native apps give more control.

For ButterShare, I decided reach was worth the constraints.

P2P changes scaling in an interesting way

In a storage-centric product, user growth often means your central infrastructure moves proportionally more file data.

In a P2P product, some of that load is distributed to the edge.

That is conceptually interesting.

Instead of:

10 users → server traffic
1,000 users → much more server traffic
100,000 users → enormous server traffic

the peers carry much of the data themselves when direct connections succeed.

Your signaling infrastructure can scale very differently from the files themselves.

This is one of the most attractive technical properties of the design.

But connection count still matters

Even if your server is not handling the bytes, you may still have thousands or millions of sessions.

You need to scale:

signaling connections
WebSockets
room state
ephemeral session data
TURN credential generation
monitoring

P2P reduces one category of scaling problem.

It does not eliminate scaling.

Why I didn't choose a hybrid storage-first architecture

A hybrid model is also possible.

For example:

Try P2P first.

If direct transfer fails:
upload to S3.

This sounds great.

But it dramatically increases product complexity.

Now you have two transfer engines:

P2P path
cloud path

And you need to answer:

When do we switch?
Does the user know?
Who pays for storage?
How long is the cloud copy retained?
Do we start uploading immediately as fallback?
What if P2P reconnects?
How does resume work across both modes?

A hybrid architecture may make sense later.

But for a focused product, simplicity of product behavior has value.

Why not automatically upload while trying P2P?

You could do both:

Sender → Receiver
   \
    → S3 backup

Then if the peer connection fails, the recipient continues from the cloud.

Very robust.

Also potentially very expensive.

You just recreated the cloud bandwidth problem even when direct transfer works.

Again, architecture follows priorities.

The simplicity I wanted was conceptual, not technical

This is a funny thing about ButterShare.

The technical implementation is complicated.

But the product concept is extremely simple:

I have a file.

You need the file.

Let's connect.

I wanted the infrastructure to reflect that as much as possible.

That was more important to me than choosing the easiest backend implementation.

What I would choose for different products

If I were starting other products today, my choices would be different.

Photo backup app

S3/object storage.

Team document workspace

S3/object storage.

Video hosting platform

Object storage + CDN.

Software distribution

Object storage + CDN.

One-time 100GB transfer between two online people

P2P becomes very interesting.

The phrase:

use the right tool for the job

is boring.

It is also correct.

A simplified architecture for ButterShare

At a high level, the system looks like this:

               +-------------------+
               | Signaling Service |
               +-------------------+
                    ↑          ↑
                    |          |
                    |          |
                    ↓          ↓
               Sender      Receiver
               Browser      Browser
                   \          /
                    \        /
                     \      /
                      \    /
                WebRTC Connection
                        |
                        |
                 Direct if possible

If direct connection cannot be established:

Sender
   |
   v
TURN relay
   |
   v
Receiver

The important part is that the file does not need to become a traditional stored object before the recipient can receive it.

The decision came down to product identity

The more I thought about it, the less this felt like a pure infrastructure decision.

It became a product decision.

If I built ButterShare around S3, the product would effectively be:

Upload files to ButterShare and share them.

With P2P, the product becomes:

Use ButterShare to connect two devices and transfer files.

Those are subtly different.

I wanted the second one.

What I gave up by not choosing S3

There are real downsides.

I gave up easy:

asynchronous downloads
persistent links
sender-offline downloads
one-to-many distribution
mature multipart infrastructure
global cloud delivery
simple browser downloading
durable server-side resume state

Those are significant benefits.

I do not pretend otherwise.

What I gained

In exchange, I gained an architecture designed around:

direct transfer
very large files
minimal persistent file storage
potentially lower central bandwidth requirements
no traditional cloud-storage quota requirement
cross-platform browser access
simple one-time transfer sessions

For ButterShare, those benefits align closely with the product.

That is why the tradeoff made sense.

The lesson I would give another developer

Do not start with:

Should I use S3 or WebRTC?

Start with:

What exactly is my product doing with the file?

Ask:

Does the file need to exist tomorrow?

Does the sender need to stay online?

Will multiple people download it?

Does it need versioning?

Is this storage or transfer?

How large are the files?

Who pays for bandwidth?

How important is app-free browser access?

The answers will usually tell you which architecture makes sense.

My decision tree

This is roughly how I think about it.

If the answer to:

Must the file remain available when the sender goes offline?

is:

Yes

use cloud storage.

If:

Does the file need to be downloaded by many people?

is:

Yes

cloud storage/CDN probably wins.

If:

Is this basically long-term storage?

is:

Yes

use object storage.

But if:

one sender
one receiver
both online
large file
one-time transfer

then P2P deserves serious consideration.

Final thoughts

Choosing P2P instead of S3 made ButterShare harder to build.

There is no question about that.

S3 would have solved many problems for me immediately.

With WebRTC, I had to care about:

peer discovery
signaling
STUN
TURN
NAT traversal
browser memory
chunking
backpressure
progressive writes
reconnection
resume
mobile browsers

That is a lot of engineering.

But it also created a product architecture that matches the problem I actually wanted to solve.

I am not trying to create another cloud drive.

I am trying to make this:

Device A
   ↓
Device B

feel as simple as possible.

That is why I chose P2P.

Not because S3 is bad.

Not because P2P is magically free.

Not because WebRTC solves everything.

But because for one-time, extremely large transfers between two online devices, sending the file directly to its destination makes a lot of sense.

And once I started thinking about file transfer that way, it became very difficult to go back to:

Upload it somewhere else first.

I'm building that idea into ButterShare, a browser-based peer-to-peer file transfer tool for sending large files directly between devices without relying on traditional cloud file storage.

ButterShare:

https://buttershare.com/

TL;DR

I chose P2P instead of building ButterShare around S3 because the product is primarily about transferring files, not storing them.

With S3:

Sender → Cloud → Receiver

With P2P:

Sender → Receiver

P2P gives me:

no requirement to permanently store every file
potentially less centralized bandwidth usage
direct device-to-device transfer
better economics for very large one-time transfers
a browser-based cross-platform workflow

But I also accept:

harder engineering
TURN costs
both users needing to be online
more browser complexity
harder resume and reliability problems

For Dropbox-style storage, I would choose object storage.

For sending a 100GB file between two people who are online right now, I think P2P is a much more interesting architecture.

WebRTC Data Channels for Large File Transfer: What I Learned the Hard Way

Anirban Roy — Wed, 23 Sep 2026 04:29:26 +0000

WebRTC looks deceptively simple when you first use it for file transfer.

Create a peer connection.

Open a data channel.

Read a file.

Send the bytes.

Done.

That approach works beautifully in a demo.

Then you try a 20GB file.

Then 50GB.

Then 100GB.

Then Safari decides your tab has had enough.

Then the receiver runs out of memory.

Then the sender's bufferedAmount keeps climbing.

Then the connection drops at 93%.

Then you realize the hard part was never getting bytes from one browser to another.

The hard part is doing it reliably, for a long time, across real browsers, on real networks, without blowing up memory or making the user restart from zero.

That is what I learned while building large-file transfer into ButterShare.

This post is not about a toy WebRTC example.

It is about the parts that start to matter when your transfer is no longer 5MB.

WebRTC Data Channels are powerful, but they are not a file transfer protocol

That was the first important lesson.

A WebRTC data channel gives you a way to move arbitrary data between peers.

That is all.

It does not automatically give you:

file metadata
chunking
progress
backpressure
retry logic
resume support
integrity checks
folder structure
acknowledgements
transfer state
error recovery

If you need those things, you have to design them.

The browser gives you transport.

You still need to build the protocol.

That difference becomes very important once transfers get large.

The naive implementation

The obvious first version looks something like this:

const buffer = await file.arrayBuffer()
dataChannel.send(buffer)

For a tiny file, this is fine.

For a large file, it is a disaster.

If the file is 10GB, you absolutely do not want to read 10GB into memory.

And you definitely do not want to do this on a phone.

The better mental model is:

Never think of a large file as one object.

Think of it as a sequence of chunks.

Chunking is not optional

The basic idea is simple.

Instead of this:

[ 100GB file ]

you treat it as:

[chunk 1]
[chunk 2]
[chunk 3]
[chunk 4]
...

In JavaScript:

const CHUNK_SIZE = 64 * 1024

let offset = 0

while (offset < file.size) {
  const chunk = file.slice(offset, offset + CHUNK_SIZE)
  const buffer = await chunk.arrayBuffer()

  dataChannel.send(buffer)

  offset += CHUNK_SIZE
}

This is already much better.

You only keep a small part of the file in memory at a time.

But this still breaks under pressure.

Why?

Because you're sending chunks faster than the network can necessarily transmit them.

That leads to the next problem.

`bufferedAmount` will humble you

WebRTC data channels have an internal outgoing buffer.

You can inspect it with:

dataChannel.bufferedAmount

If you keep doing this:

dataChannel.send(chunk)

inside a fast loop, you can fill that buffer faster than the browser can drain it.

At first, everything looks normal.

Then memory starts climbing.

Then the tab becomes sluggish.

Then performance collapses.

Then the browser may kill the page.

This is one of the first things that separates a demo from a serious file-transfer implementation.

Backpressure matters more than chunk size

A common mistake is focusing too much on finding the "perfect" chunk size.

Should it be:

16KB?
64KB?
256KB?
1MB?

Chunk size matters, but flow control matters more.

You need to slow down when the browser's send buffer gets too large.

For example:

const MAX_BUFFERED = 4 * 1024 * 1024

async function waitForBuffer() {
  while (dataChannel.bufferedAmount > MAX_BUFFERED) {
    await new Promise(resolve => setTimeout(resolve, 10))
  }
}

Then:

while (offset < file.size) {
  await waitForBuffer()

  const chunk = file.slice(offset, offset + CHUNK_SIZE)
  const buffer = await chunk.arrayBuffer()

  dataChannel.send(buffer)

  offset += CHUNK_SIZE
}

That works better.

But polling every few milliseconds is still not ideal.

WebRTC gives you a cleaner option.

Use `bufferedAmountLowThreshold`

You can tell the browser:

Let me know when your send buffer has dropped below this level.

For example:

dataChannel.bufferedAmountLowThreshold = 1024 * 1024

Then:

function waitUntilWritable(channel) {
  if (channel.bufferedAmount <= channel.bufferedAmountLowThreshold) {
    return Promise.resolve()
  }

  return new Promise(resolve => {
    const handler = () => {
      channel.removeEventListener("bufferedamountlow", handler)
      resolve()
    }

    channel.addEventListener("bufferedamountlow", handler)
  })
}

Now the sender can wait until the channel is ready for more data.

This is far more stable than dumping chunks into the buffer as fast as possible.

Large file transfer is a flow-control problem

Once files get big, your job is not:

send as fast as possible

It is:

send as fast as the browser and network can safely sustain

That distinction matters.

If you push too aggressively, you get:

memory pressure
unstable throughput
browser crashes
poor mobile behavior

If you throttle too much, you leave bandwidth unused.

So you are constantly balancing:

speed
vs
stability

And for 100GB transfers, stability wins.

A slightly slower transfer that finishes is better than a faster transfer that dies after 70GB.

Metadata has to come first

The receiver needs context before binary chunks arrive.

At minimum, I want the receiver to know things like:

{
  "type": "file-start",
  "id": "file-001",
  "name": "final-cut.mov",
  "size": 21474836480,
  "mimeType": "video/quicktime"
}

Then the actual file data can follow.

For multiple files:

{
  "type": "transfer-start",
  "files": [
    {
      "id": "file-001",
      "name": "video.mov",
      "size": 21474836480
    },
    {
      "id": "file-002",
      "name": "thumbnail.jpg",
      "size": 5278321
    }
  ]
}

Now the receiver knows what to expect.

This sounds trivial.

It becomes very important for:

progress calculation
folder reconstruction
resume
error reporting
multi-file transfers

I stopped thinking in messages and started thinking in protocol states

At first, you can get away with random messages like:

"start"
"chunk"
"done"

That gets messy very quickly.

A proper protocol is easier to reason about.

For example:

SESSION_READY
TRANSFER_START
FILE_START
FILE_CHUNK
FILE_PROGRESS
FILE_END
TRANSFER_END
ACK
RETRY
RESUME_REQUEST
ERROR

Once you have that, the system becomes much easier to debug.

You can actually answer:

What state is the receiver in?

instead of:

Why is this boolean false?

The receiver is where many implementations fall apart

Sending chunks safely is only half of the problem.

A lot of examples do this on the receiver:

const chunks = []

channel.onmessage = event => {
  chunks.push(event.data)
}

Then:

const blob = new Blob(chunks)

This is fine for small files.

It is completely wrong for huge files.

If you're receiving 100GB and storing every chunk in memory, you are effectively trying to build a 100GB array inside the browser.

That is not going to end well.

The receiver needs progressive writing.

Progressive writing is where browser differences become painful

On supported browsers, APIs such as the File System Access API can let you write incrementally.

Conceptually:

const handle = await window.showSaveFilePicker({
  suggestedName: fileName
})

const writable = await handle.createWritable()

Then as chunks arrive:

await writable.write(chunk)

And finally:

await writable.close()

This is the right model.

Data arrives.

Data is written.

Memory stays bounded.

But browser support is not uniform.

That is where the product challenge starts.

Chrome on desktop may give you capabilities that Safari on iPhone does not expose in the same way.

So you cannot build your architecture assuming one browser API is universally available.

"Works in Chrome" means almost nothing

One of the biggest mistakes in browser networking is testing only in Chrome desktop.

A real file-transfer product needs to survive:

Chrome
Edge
Firefox
Safari
iOS Safari
Android Chrome
low-memory devices
aggressive mobile OS background behavior

And each one behaves differently under long transfers.

Desktop WebRTC is the easy part.

Mobile is where the real problems show up.

Mobile browsers are brutal

A 30-minute transfer on desktop might be fine.

On a phone, many things can interrupt you:

screen locking
switching apps
battery saver
background tab throttling
memory pressure
network handoff
Wi-Fi to mobile-data switching
browser suspension

The user does not care why.

They just see:

Transfer failed.

That means your UX needs to prepare them.

For example:

Keep this tab open until the transfer finishes.

Or:

Avoid locking your phone during large transfers.

It may not feel elegant, but it is better than pretending the browser has infinite control over the operating system.

Connection setup is not the same as file transfer

WebRTC needs signaling before the peers can communicate.

Usually that means exchanging:

SDP offer
SDP answer
ICE candidates

Your signaling server helps both sides find each other.

Something like:

Sender
  |
  | Offer
  v
Signaling server
  |
  | Forward
  v
Receiver

Then:

Receiver
  |
  | Answer
  v
Signaling server
  |
  | Forward
  v
Sender

ICE candidates are exchanged too.

Once the peer connection is established, the file data can flow.

The key architectural point is this:

The signaling server coordinates the connection. It does not need to store the file.

That keeps the control plane separate from the data plane.

STUN and TURN are where the clean P2P story gets messy

In the ideal case:

Browser A <---------> Browser B

Direct connection.

Nice.

But users are behind:

routers
NAT
carrier-grade NAT
enterprise firewalls
VPNs
restrictive mobile networks

So WebRTC uses ICE.

STUN helps peers discover network information.

TURN is used when a direct route cannot be established.

Then you get:

Browser A
    |
    v
TURN server
    |
    v
Browser B

That transfer is still not being stored like a normal cloud-hosted file.

But the traffic is relayed.

And that matters financially.

If a 100GB transfer goes through TURN, you are now relaying 100GB.

Do that often enough and your infrastructure bill becomes very real.

P2P does not mean zero infrastructure cost

This is a misconception I see a lot.

You still need:

signaling
TURN
STUN
session management
monitoring
abuse prevention
analytics
connection diagnostics

And TURN bandwidth can be expensive.

A useful product needs to track:

direct connection rate
relay rate
average file size
average transfer duration
failure rate
resume rate
browser distribution

Otherwise, you are building blind.

Acknowledgements make progress more honest

Another thing I learned:

Do not call send() and immediately assume those bytes reached the receiver.

This:

dataChannel.send(chunk)
bytesSent += chunk.byteLength

only tells you that the browser accepted the data into its sending pipeline.

It does not necessarily mean the receiver has written it.

For a better progress model, the receiver can periodically acknowledge bytes received.

Example:

{
  "type": "ack",
  "fileId": "file-001",
  "receivedBytes": 5368709120
}

Now the sender knows:

5GB confirmed on receiver

That is much more meaningful than:

5GB queued locally

Don't acknowledge every chunk

That sounds tempting.

But if you send an ACK for every 64KB chunk, you create unnecessary protocol chatter.

Instead, acknowledge periodically.

For example:

every 4MB
every 16MB
every 100 chunks
every N milliseconds

The exact strategy depends on your implementation.

You want enough acknowledgement data to support:

accurate progress
resume
fault recovery

without turning the protocol into an ACK storm.

Resume support changes everything

If you're transferring 500MB, restarting is annoying.

If you're transferring 100GB, restarting is unacceptable.

Imagine this:

Transferred: 96.7GB
Connection lost.
Start again?

Nobody wants that.

Resume support means both sides need to agree on what has already been completed.

For example:

{
  "type": "resume-state",
  "fileId": "file-001",
  "receivedBytes": 103809024000
}

Then the sender resumes from that offset.

Conceptually:

const remaining = file.slice(receivedBytes)

Of course, a production system needs more checks than this.

You need to ensure:

same file
same session
same size
correct offset
receiver state is valid

But the concept is straightforward.

Resume is not a nice extra for huge files.

It is part of the core product.

File identity matters for resume

You cannot blindly trust:

file name

Two files can have the same name.

So a real resume mechanism should use better identity information.

Possibilities include:

file size
last-modified timestamp
transfer-generated ID
checksum
chunk hashes

For example:

{
  "id": "transfer-file-8f73",
  "name": "footage.mov",
  "size": 82471843840,
  "lastModified": 1788402394000
}

For stronger verification, add a digest.

The exact level depends on your threat model and performance requirements.

Hashing huge files has a cost too

Hashing a 100GB file before transfer sounds great.

Until the user waits while you read the entire 100GB once just to calculate the hash.

Then they wait again while you actually send it.

That may be unacceptable.

A better strategy can be incremental.

For example:

hash while transferring
hash chunks
hash groups of chunks
calculate a rolling digest

This gives you integrity verification without requiring a full extra pass before transfer.

Large file transfer makes small bugs expensive

A 0.1% edge case sounds tiny.

At scale, it is not.

And for a user in the middle of a long transfer, that one bug can waste an hour.

With huge transfers, things that seem minor suddenly matter:

duplicate chunks
missing chunks
out-of-order logic
reconnect races
incorrect offsets
final partial chunk handling
file-close timing
stale session state

A 20KB mistake at the end of a 100GB file is still a corrupted file.

Ordered vs unordered data channels

WebRTC data channels can be configured in different ways.

The default is ordered delivery.

For file transfer, ordered delivery is usually the simplest choice because file chunks naturally belong in sequence.

For example:

const channel = peerConnection.createDataChannel("file-transfer", {
  ordered: true
})

Unordered delivery can be useful for other real-time use cases.

For a file-transfer system, ordered delivery generally makes reconstruction easier.

That said, your application-level protocol should still know which chunk or byte range it is dealing with.

Do not rely entirely on transport behavior as your only source of truth.

Reliable vs partially reliable delivery

WebRTC lets you configure data channels with options such as:

maxRetransmits

or:

maxPacketLifeTime

Those are useful for real-time data where dropping an old packet can be better than waiting.

Examples:

games
live telemetry
fast-changing state

File transfer is different.

If a chunk disappears, the file becomes corrupt.

For file transfer, reliability usually matters more than latency.

You do not want:

99.999% of the file

You want:

100%

Chunk size is a tradeoff

There is no universal best chunk size.

Small chunks:

Pros

lower memory pressure
more granular progress
easier retry boundaries

Cons

more application overhead
more messages
more protocol bookkeeping

Large chunks:

Pros

less message overhead
potentially better throughput

Cons

larger temporary memory
less granular resume
more painful retries

For many implementations, something in the tens of kilobytes is a reasonable place to start.

But benchmark on your target browsers.

Do not copy a chunk size from a blog post and assume it is correct for your product.

Including this one.

Throughput is not just network bandwidth

Let's say the sender has:

1 Gbps connection

That does not mean you will get a 1 Gbps browser transfer.

Other bottlenecks may include:

Wi-Fi
CPU
encryption overhead
JavaScript execution
disk read speed
disk write speed
browser implementation
TURN path
packet loss
mobile hardware

You need to measure the real end-to-end pipeline.

A file transfer is roughly:

disk read
   ↓
JavaScript
   ↓
WebRTC buffer
   ↓
network
   ↓
WebRTC receiver
   ↓
JavaScript
   ↓
disk write

The slowest part wins.

Local network transfers can be extremely fast

One thing that makes peer-to-peer transfer interesting is that nearby devices may be able to communicate efficiently over the local network.

If two capable machines are on fast Wi-Fi or Ethernet, local transfers can perform very well.

But again, there are no guarantees.

Network topology matters.

Browsers matter.

Routing matters.

You should never hardcode claims like:

always gigabit speed

unless you can actually guarantee that.

You usually cannot.

A better claim is:

Performance depends on the connection between the devices.

Less exciting.

More accurate.

Error messages matter

Developers love errors like:

ICE connection state failed

Users do not.

They need:

Could not establish a direct connection. Retrying...

Or:

The transfer was interrupted. Reconnecting from 42.8GB...

Or:

Your device went offline. Check your connection and try again.

A good transfer product converts networking details into understandable states.

Connection state is more nuanced than connected/disconnected

You will see states like:

new
checking
connected
completed
disconnected
failed
closed

And disconnected does not always mean permanently dead.

A network switch may briefly cause disconnects.

That means your app should not immediately destroy the session when the connection flickers.

You need grace periods.

Retries.

State transitions.

This is why I eventually stopped using simple flags and started thinking in terms of a state machine.

A transfer state machine is worth it

Something like:

IDLE
  ↓
CREATING_ROOM
  ↓
WAITING_FOR_PEER
  ↓
CONNECTING
  ↓
READY
  ↓
TRANSFERRING
  ↓
COMPLETED

With side paths:

TRANSFERRING
  ↓
DISCONNECTED
  ↓
RECONNECTING
  ↓
RESUMING
  ↓
TRANSFERRING

And failure:

RECONNECTING
  ↓
FAILED

This structure makes your UI, analytics, and error recovery much cleaner.

It is much better than managing:

isSending
isConnected
isReconnecting
isDone
hasError

and hoping every combination makes sense.

Don't tie your protocol directly to React state

This one is more architectural.

Your transfer engine should not exist as a pile of React state and hooks.

The transfer process can live for:

minutes
hours

It deserves its own abstraction.

I prefer separating:

UI
Protocol
Transport
File IO
Session state

For example:

React UI
   ↓
Transfer Controller
   ↓
Transfer Protocol
   ↓
WebRTC Transport

That makes debugging much easier.

It also means the networking logic is not constantly being rewritten because the UI changed.

Multiple files are harder than one huge file

A single 100GB file has one nice property:

It is one continuous byte range.

A folder containing 20,000 files is more complicated.

Now you need:

file boundaries
metadata
directory structure
relative paths
empty directories
duplicate names
skipped files
per-file progress

The protocol becomes something like:

TRANSFER_START

FILE_START
CHUNKS...
FILE_END

FILE_START
CHUNKS...
FILE_END

...

TRANSFER_END

For folders, you also need path information.

Example:

{
  "type": "file-start",
  "name": "hero.png",
  "relativePath": "project/assets/images/hero.png",
  "size": 834021
}

Now the receiver can reconstruct the structure.

Sending 20,000 tiny files can be worse than one huge file

This is another surprise.

A 50GB single video may be easier to handle than:

50GB spread across 100,000 tiny files

Why?

Because every file adds overhead:

metadata
open/close operations
filesystem work
protocol messages
progress updates

This is why bundling directories can sometimes make sense.

But then you have tradeoffs around:

ZIP creation time
memory
browser support
streaming archives

Nothing is free.

The browser is capable, but not unlimited

I love building serious applications in the browser.

But it is important to respect the environment.

A browser tab is not a native daemon.

It can be:

suspended
throttled
killed
reloaded
closed accidentally

Your design needs to assume imperfect conditions.

That means:

save session state where reasonable
support reconnection
avoid huge in-memory structures
communicate clearly with the user
test long-running transfers

Test duration, not just file size

This is a subtle one.

You may test a 50GB transfer on a fast LAN and finish quickly.

Then assume 50GB works.

But a remote user with slow upload might take several hours.

That means your system also needs to survive:

long duration

not just:

large byte count

Those are different stress tests.

I would test scenarios like:

5GB over fast LAN
50GB over fast LAN
5GB over slow internet
50GB over throttled internet
2-hour transfer
4-hour transfer
mobile sender
mobile receiver
network interruption mid-transfer

You learn different things from each one.

Simulate bad networks

Do not test only on your office Wi-Fi.

Large-transfer software needs to survive:

latency
packet loss
slow uploads
unstable Wi-Fi
connection changes

Use network throttling where possible.

Switch networks manually.

Turn Wi-Fi off temporarily.

Lock the phone.

Background the browser.

Try the things real users will accidentally do.

The bugs you find this way are usually much more valuable than another perfect localhost test.

TURN fallback needs real-world testing

It is easy to say:

TURN works.

You need to test:

different NAT types
corporate networks
VPNs
mobile carriers
IPv4
IPv6

And then monitor how often TURN is actually used.

If your product starts growing and 40% of huge transfers are going through TURN, your cost model looks very different from what you expected.

Security is more than encrypted transport

WebRTC uses encrypted transport, which is great.

But your application still needs to think about:

room guessing
unauthorized joining
expired sessions
signaling authentication
rate limiting
replay
transfer links
metadata exposure

If your room code is:

ABC123

how hard is it to guess?

How many attempts can someone make?

How long does that room exist?

Does the share URL include additional entropy?

Security is a system property.

You cannot just say:

WebRTC is encrypted, therefore we're done.

Keep transfer sessions short-lived

For a real-time transfer product, there is rarely a reason for room/session state to live forever.

Short-lived sessions reduce:

stale data
attack surface
database clutter
accidental reuse

A session can expire after inactivity.

Once both peers leave, clean it up.

This keeps the coordination layer simple.

Observability becomes essential

When users tell you:

It stopped at 67%.

that is not enough information.

You need operational data that helps diagnose the failure.

Useful telemetry might include:

browser
browser version
OS
connection type
peer state transitions
ICE state
candidate type
direct vs relay
transfer size
bytes completed
transfer duration
error stage

You do not need to inspect users' files.

You do need to understand why the transport failed.

Candidate type is useful to measure

Knowing whether the transfer used:

host
srflx
relay

can help explain performance.

If poor-performing sessions are mostly:

relay

that tells you something.

If failures are concentrated in one browser version, that tells you something else.

Data helps you avoid guessing.

Success rate matters more than benchmark speed

It is tempting to market:

500 Mbps transfer

But the metric I care about more is:

What percentage of transfers actually complete?

I would rather have:

95% completion at 200 Mbps

than:

70% completion at 600 Mbps

For huge files, reliability is the product.

Speed only matters after reliability.

What I learned about "no file size limit"

Technically, with a streaming architecture, you do not need a traditional server-side file-size cap because the full file is not sitting in your application memory or object storage.

But that does not mean there are no practical limits.

The real constraints become:

browser capabilities
device storage
device memory
network stability
transfer duration
filesystem support

So the phrase:

no file size limit

should mean:

the application does not impose an arbitrary product cap

not:

physics and device limitations no longer exist

That distinction is important.

Large transfer UX needs constant reassurance

A 100GB transfer can take a long time.

Users need confidence that something is happening.

Good UI should show:

bytes transferred
percentage
current speed
estimated time remaining
connection state
sender/receiver status

Something like:

47.8 GB of 100 GB
47.8%
18.4 MB/s
~47 minutes remaining

That feels trustworthy.

A spinner does not.

ETA is harder than it looks

If you calculate ETA from the current instant speed:

eta = remainingBytes / currentSpeed

it jumps all over the place.

Better to smooth throughput over time.

For example, using a rolling average.

Conceptually:

last 10 seconds
last 30 seconds
exponential moving average

That produces a much calmer ETA.

Nobody likes:

12 minutes remaining
38 minutes remaining
8 minutes remaining
51 minutes remaining

every few seconds.

Progress should be receiver-confirmed when possible

Again, this matters for trust.

If the sender has queued 80GB but the receiver has only written 75GB, showing:

80%

may be misleading.

For long transfers, I prefer basing progress as much as possible on confirmed receiver state.

That becomes especially valuable for resumability.

Don't optimize too early

There is a temptation to start with:

parallel channels
adaptive chunking
compression
exotic scheduling
custom congestion tricks

Before doing that, get the fundamentals right:

stable connection
bounded memory
backpressure
progressive writing
accurate state
reconnect
resume
integrity

Only then optimize.

A boring transfer engine that finishes is better than a clever one that breaks.

When parallel data channels might help

It is tempting to open multiple data channels and send parts of the file concurrently.

That can sometimes improve throughput in specific situations.

But it also creates more complexity:

ordering
reassembly
congestion
memory
debugging
resume state

I would not start there.

One well-managed reliable channel can go a long way.

Optimize only after measurements show a real bottleneck.

WebRTC is not always the right answer

This is important.

If the sender does not need to stay online, WebRTC is probably not the right architecture.

If the recipient needs to download tomorrow, use storage.

If thousands of people need the same file, use a CDN.

If the file needs permanent access, use object storage.

WebRTC makes the most sense when:

one sender
one receiver
both online
large file
real-time transfer

That is the sweet spot.

The architecture I prefer

At a high level:

        Signaling Server
          /        \
         /          \
        /            \
   Sender           Receiver
   Browser           Browser
      \               /
       \             /
        \           /
        WebRTC Channel
              |
              |
       Direct if possible
              |
              v
        File transfer

With TURN when necessary:

Sender
   |
   v
TURN Relay
   |
   v
Receiver

And the application protocol sits above the transport:

Transfer Protocol
      |
      v
WebRTC Transport
      |
      v
Network

That separation makes everything cleaner.

What ButterShare taught me

Building ButterShare changed how I think about browser networking.

At first, I thought:

WebRTC can send binary data. Great. File transfer solved.

Now I think:

WebRTC gives me a transport primitive. Everything else is engineering.

The hardest parts were not:

new RTCPeerConnection()

or:

createDataChannel()

The hard parts were:

controlling memory
handling backpressure
surviving long transfers
supporting mobile browsers
recovering from disconnects
making progress trustworthy
keeping the protocol understandable
avoiding cloud storage while still handling real-world network failures

That is where the real work is.

If I were building it again

I would do these things from day one:

1. Build a transfer protocol first

Define the message types before wiring everything into the UI.

2. Treat backpressure as a first-class feature

Not an optimization.

A requirement.

3. Design resume early

Do not wait until users complain about losing 80GB of progress.

4. Write progressively on the receiver

Never assume the entire file can fit in memory.

5. Test mobile early

Desktop success will give you false confidence.

6. Measure direct vs relayed transfers

TURN usage directly affects cost and performance.

7. Build a state machine

Large-transfer state becomes too complex for a handful of booleans.

8. Separate UI from transport

Do not make networking logic depend on component lifecycle.

9. Track receiver-confirmed progress

It makes resume and UX much better.

10. Optimize only after reliability

A finished transfer beats a benchmark screenshot.

Final thoughts

WebRTC Data Channels are one of the most interesting browser technologies for direct file transfer.

They let you build something that would have required native applications not that long ago.

But the API itself is the easy part.

The moment you move beyond small demo files, you start dealing with real systems problems:

flow control
memory
reliability
networking
browser behavior
state management
failure recovery
observability

That is exactly why I enjoy working on it.

The user sees:

Send file

The engineering underneath sees:

discover peer
negotiate connection
traverse NAT
manage buffer
read chunks
send chunks
ack chunks
write chunks
track state
handle disconnect
reconnect
resume
verify
complete

That is the gap between a demo and a product.

I am building these ideas into ButterShare, a browser-based peer-to-peer file transfer tool focused on moving large files directly between devices without relying on traditional cloud file storage.

If you have worked on WebRTC Data Channels, large-file streaming, or browser networking, I would be interested to hear which problem caused you the most pain.

ButterShare:

https://buttershare.com/

TL;DR

If you want to use WebRTC Data Channels for large file transfers:

Do not load the whole file into memory.
Chunk the file.
Respect bufferedAmount.
Use bufferedAmountLowThreshold.
Build backpressure into the sender.
Write progressively on the receiver.
Design a real transfer protocol.
Use acknowledgements.
Make resume possible.
Expect TURN.
Test mobile browsers early.
Track completion rate, not just speed.
Treat 100GB transfers as long-running sessions, not oversized attachments.

The biggest lesson?

WebRTC can move the bytes. Your job is making sure the transfer survives everything that happens around those bytes.

How I Send 100GB+ Files Directly Between Browsers Without Cloud Storage

Anirban Roy — Wed, 23 Sep 2026 04:24:40 +0000

Most file-sharing products are built around the same basic idea:

Upload the file to a server.
Store it somewhere.
Generate a link.
Let the recipient download it later.

That model is simple, reliable, and works extremely well for a lot of products.

But it also creates a problem the moment you start dealing with very large files.

A 20MB PDF is easy.

A 500MB video is manageable.

A 5GB project starts to become noticeable.

At 50GB, 100GB, or more, suddenly everything becomes about storage quotas, upload times, bandwidth bills, server infrastructure, expired links, and paid plans.

I kept coming back to one question:

If the file already exists on one computer and the final destination is another computer, why does the entire file need to live on my server in the middle?

That question eventually became the foundation for ButterShare.

ButterShare is a browser-based peer-to-peer file transfer product. Instead of uploading the entire file to cloud storage first, the two browsers establish a connection and transfer the file directly.

The basic architecture is:

Traditional file sharing:

Sender
   |
   v
Cloud Storage
   |
   v
Receiver

ButterShare is closer to:

Sender
   |
   v
Receiver

Of course, the real implementation is more complicated than that diagram suggests.

There is signaling.

There is WebRTC.

There are NATs and firewalls.

There is buffering.

There is flow control.

There are browser memory limits.

There are mobile devices that decide your background tab is no longer important.

And there is the very uncomfortable question:

What happens when a 100GB transfer fails at 97%?

This post is about what I learned while designing large browser-to-browser file transfer and why sending 100GB directly between browsers is a very different problem from sending a normal attachment.

The first realization: file transfer and file storage are different products

This sounds obvious, but a lot of file-sharing software combines the two.

When someone says:

“Send this file to John.”

there are at least two possible interpretations.

Storage-based sharing

The sender uploads the file somewhere.

The server keeps it.

John downloads it whenever he wants.

This is asynchronous.

The sender can close their laptop and disappear.

Direct transfer

The sender and John are both online.

The file moves from one device to the other.

Once the transfer is complete, the infrastructure does not need to retain the actual file.

This is synchronous.

They solve different problems.

Cloud storage is better when persistence matters.

Direct transfer becomes interesting when both people are online and the only objective is moving data from A to B.

For large files, removing the storage layer changes a lot.

Why 100GB changes the architecture

Imagine a user wants to send a 100GB file.

With a conventional cloud architecture, the file usually crosses your infrastructure twice.

First:

Sender -> Your server/storage

Then:

Your server/storage -> Receiver

That means a single 100GB transfer can result in roughly 200GB of file traffic moving through infrastructure you operate.

And you still have to store the file somewhere in between.

Now multiply that.

100 users transferring 100GB:

10 TB uploaded
10 TB downloaded

1,000 users:

100 TB uploaded
100 TB downloaded

Suddenly your “simple file sharing app” becomes a storage and bandwidth business.

That can absolutely be the correct business model.

But it was not the model I wanted for ButterShare.

I wanted the infrastructure to coordinate the connection without becoming the permanent warehouse for the actual file.

Why WebRTC fits this problem

WebRTC is usually associated with video calls.

But underneath the video/audio features is something extremely useful for file transfer:

WebRTC Data Channels.

A data channel lets two peers exchange arbitrary data.

That can be:

text
JSON
binary data
file chunks
application messages

So instead of streaming video frames between browsers, you can stream pieces of a file.

Very simplified:

const peerConnection = new RTCPeerConnection()

const channel = peerConnection.createDataChannel("file-transfer")

channel.onopen = () => {
  console.log("Peer connected")
}

channel.onmessage = (event) => {
  // receive file data
}

The interesting part is that once the peer connection is established, the data can often travel directly between the two devices.

Your application server does not need to handle each file chunk.

That is the key architectural difference.

But WebRTC does not magically connect two browsers

Before two browsers can communicate, they have to find each other.

This is where signaling comes in.

The signaling server exchanges connection metadata between the peers.

Typical information includes:

SDP offers
SDP answers
ICE candidates

The rough process looks like this:

Sender Browser
     |
     | create offer
     v
Signaling Server
     |
     | forward offer
     v
Receiver Browser

Receiver Browser
     |
     | create answer
     v
Signaling Server
     |
     | forward answer
     v
Sender Browser

Both sides also exchange ICE candidates.

Eventually WebRTC attempts to establish the actual peer connection.

The important distinction is:

The signaling server helps the devices connect. It does not necessarily carry the file itself.

That distinction matters enormously when the file is 100GB.

A few KB of signaling traffic is trivial.

100GB of file traffic is not.

STUN, TURN, and the part everyone leaves out

This is where the nice architecture diagram starts becoming less clean.

You cannot assume every pair of devices can establish a perfect direct connection.

Users sit behind:

routers
NAT
carrier-grade NAT
corporate networks
restrictive firewalls
mobile networks
VPNs

WebRTC uses ICE to figure out how the peers can communicate.

A STUN server helps a device discover its public-facing network information.

In ideal conditions:

Browser A <----------------> Browser B

Direct.

Beautiful.

Cheap.

Fast.

But sometimes direct connectivity is impossible.

That is where a TURN server becomes important.

A TURN server relays traffic:

Browser A
    |
    v
TURN Relay
    |
    v
Browser B

At that point, your infrastructure may be handling the traffic again.

The important difference is that relaying encrypted traffic is not the same thing as permanently storing the file.

The relay forwards packets.

It does not need to create a downloadable cloud copy of the file.

Still, from an infrastructure perspective, TURN traffic is real bandwidth and needs to be treated seriously.

So “peer-to-peer” does not mean:

100% of every transfer will always bypass every server.

It means:

The system attempts direct peer connectivity and can avoid traditional cloud file storage.

That is a much more accurate description.

The naive implementation fails immediately

When developers first build browser file transfer, the natural approach often looks something like this:

const buffer = await file.arrayBuffer()
dataChannel.send(buffer)

This is fine for small files.

It is a terrible idea for 100GB files.

If the file is 100GB, you obviously do not want to load 100GB into browser memory.

Even far smaller files can create problems.

The browser may:

freeze
consume huge amounts of memory
kill the tab
become unresponsive
crash on mobile

Large file transfer needs to be streaming-oriented.

The file should be processed incrementally.

Chunking the file

Instead of treating the file as one huge object, divide it into manageable chunks.

Conceptually:

100GB file

[chunk 1]
[chunk 2]
[chunk 3]
[chunk 4]
...
[chunk 1,500,000]

The exact number depends on your chunk size.

A simplified sender might look like this:

const CHUNK_SIZE = 64 * 1024

let offset = 0

while (offset < file.size) {
  const chunk = file.slice(offset, offset + CHUNK_SIZE)
  const buffer = await chunk.arrayBuffer()

  dataChannel.send(buffer)

  offset += CHUNK_SIZE
}

This solves one major problem:

You no longer need the whole file in memory.

But this code still has another serious problem.

It sends data as fast as JavaScript can produce it.

The network cannot necessarily keep up.

Backpressure is one of the most important parts

A WebRTC data channel has an internal send buffer.

You can inspect it using:

dataChannel.bufferedAmount

If you continuously call:

dataChannel.send(...)

faster than the connection can transmit data, the buffer keeps growing.

Eventually you can create:

huge memory usage
poor performance
browser instability
transfer failure

So you need backpressure.

Instead of endlessly pushing data, you wait when the channel buffer gets too large.

Something like:

const MAX_BUFFER = 4 * 1024 * 1024

async function waitForBuffer() {
  while (dataChannel.bufferedAmount > MAX_BUFFER) {
    await new Promise(resolve => setTimeout(resolve, 10))
  }
}

Then:

while (offset < file.size) {
  await waitForBuffer()

  const chunk = file.slice(offset, offset + CHUNK_SIZE)
  const buffer = await chunk.arrayBuffer()

  dataChannel.send(buffer)

  offset += CHUNK_SIZE
}

A better implementation can use:

bufferedAmountLowThreshold

and:

onbufferedamountlow

rather than polling.

Conceptually:

dataChannel.bufferedAmountLowThreshold = 1024 * 1024

function waitUntilWritable() {
  if (dataChannel.bufferedAmount <= dataChannel.bufferedAmountLowThreshold) {
    return Promise.resolve()
  }

  return new Promise(resolve => {
    dataChannel.onbufferedamountlow = () => resolve()
  })
}

This is one of the biggest differences between a demo file-transfer project and something designed for genuinely large files.

Metadata comes before the bytes

The receiver needs to know what is coming.

Before transmitting file chunks, send metadata.

For example:

{
  "type": "file-start",
  "name": "project-final.mov",
  "size": 107374182400,
  "mimeType": "video/quicktime"
}

The receiver can then prepare for the incoming stream.

You may also send information such as:

{
  "id": "file_123",
  "relativePath": "project/videos/project-final.mov",
  "chunkSize": 65536,
  "totalChunks": 1638400
}

This becomes particularly important when sending folders containing multiple files.

The transport layer should understand the difference between:

file metadata

and:

binary file chunk

and:

transfer complete

A simple message protocol might include:

TRANSFER_START
FILE_START
FILE_CHUNK
FILE_END
TRANSFER_END
ERROR
ACK

Once you start thinking in terms of a protocol rather than random WebRTC messages, the system becomes much easier to reason about.

Why 100GB means you need BigInt thinking

JavaScript numbers can safely represent integers up to:

9,007,199,254,740,991

So 100GB itself is nowhere near the safe-integer limit.

But large-file systems still benefit from being disciplined about offsets, byte counts, chunk indexes, and serialization.

You do not want something like this hidden deep inside your protocol:

progress = chunksReceived * chunkSize

without considering the final partial chunk, reconnect offsets, multiple files, and aggregated folder size.

For real transfers, I prefer tracking actual byte counts:

bytesSent += chunk.byteLength

Then progress becomes:

const percentage = (bytesSent / totalBytes) * 100

Simple, but reliable.

Progress bars lie if you build them badly

A file-transfer progress bar looks trivial.

It is not.

Suppose you have sent data into the WebRTC buffer.

Does that mean:

JavaScript processed it?
the browser queued it?
the network transmitted it?
the receiver received it?
the receiver wrote it to disk?

Those are different stages.

If you update progress immediately after:

dataChannel.send(chunk)

you are measuring how much data you queued, not necessarily how much arrived.

For small files, nobody notices.

For a 100GB transfer, the difference can become substantial.

A more reliable system can use receiver acknowledgements.

For example:

Sender:
chunk 1000 sent

Receiver:
chunk 1000 received

Sender:
update confirmed progress

You probably do not want an acknowledgement for every tiny chunk because that adds overhead.

Instead, acknowledge ranges or byte offsets periodically.

Example:

{
  "type": "ack",
  "receivedBytes": 5368709120
}

Now the sender knows the receiver has confirmed roughly 5GB.

That is much more meaningful.

The receiver has the same memory problem

Chunking the sender is only half the solution.

A naive receiver might do this:

const chunks = []

channel.onmessage = event => {
  chunks.push(event.data)
}

const blob = new Blob(chunks)

Again, fine for small files.

Terrible for 100GB.

You have just moved the memory problem from the sender to the receiver.

You do not want to accumulate 100GB of chunks in RAM.

For truly large files, the receiver needs a way to write progressively.

Modern browser capabilities can make this possible, but support varies.

Depending on the browser and platform, you may use mechanisms such as:

File System Access API
writable file streams
browser-specific download strategies
temporary storage
service workers

This is where browser compatibility becomes a major design constraint.

Desktop Chromium gives you capabilities that mobile Safari may not.

If your product promises cross-platform transfer, you cannot design only for Chrome on your MacBook.

Desktop browsers are easy mode compared with mobile

Large file transfer on desktop is one problem.

Large file transfer on phones is another.

Mobile browsers have much tighter constraints around:

memory
storage
background execution
battery
tab suspension
screen locking
operating-system lifecycle

A user can start a transfer and then:

lock their phone
switch apps
put the browser in the background
receive a phone call
enter low-power mode

And suddenly your beautiful WebRTC connection is gone.

This creates UX requirements you do not think about in a desktop-only prototype.

For example:

Keep this tab open until the transfer finishes.

That message may sound primitive, but users need to understand the synchronous nature of peer-to-peer transfer.

You also need to handle disconnects gracefully instead of pretending they will never happen.

The hardest question: what happens when the connection dies?

Imagine transferring 100GB.

You reach:

97.4GB

Then Wi-Fi drops.

If your solution is:

“Please start again.”

the user is going to hate you.

For genuinely large files, resumability becomes extremely valuable.

That means the receiver and sender need to agree on how much data has already been transferred.

Conceptually:

{
  "fileId": "abc123",
  "receivedBytes": 104689827840
}

After reconnection:

Sender:
Okay, you already have 104,689,827,840 bytes.

Resume from there.

Then instead of:

file.slice(0)

you resume from:

file.slice(receivedBytes)

That sounds easy.

It becomes more complicated with:

multiple files
folders
changed files
chunk verification
interrupted writes
reconnecting to the wrong session

Once again, what started as a file-transfer feature becomes a protocol-design problem.

File integrity matters

A transfer showing “100%” does not necessarily mean the resulting file is correct.

Networks and transport protocols are reliable at lower layers, but once you build your own application-level chunking, reconnection, writing, and resume logic, verification becomes valuable.

One option is hashing.

Conceptually:

Sender hash:
abc123...

Receiver hash:
abc123...

If they match, great.

For huge files, hashing the entire file before transfer can itself take time.

So you may prefer chunk-level hashing or incremental hashing.

For example:

Chunk 1 -> hash
Chunk 2 -> hash
Chunk 3 -> hash

or calculate a streaming digest while data moves.

The right choice depends on how much integrity protection your product needs versus the CPU and complexity cost.

Folder transfer introduces another layer

Sending one 100GB file is actually simpler than sending a folder containing:

30,000 files

even if the folder is only 20GB.

Now you need to preserve:

filenames
relative paths
directory structure
file ordering
metadata

Conceptually:

project/
├── assets/
│   ├── logo.png
│   └── hero.jpg
├── video/
│   └── final.mov
└── project.json

You cannot just send bytes.

You need a manifest.

Something like:

{
  "files": [
    {
      "path": "assets/logo.png",
      "size": 204812
    },
    {
      "path": "assets/hero.jpg",
      "size": 5410289
    },
    {
      "path": "video/final.mov",
      "size": 42342353433
    }
  ]
}

Now the receiver knows how to reconstruct the directory structure.

This is why “send folder” is not just “send file but bigger.”

Encryption

One advantage of WebRTC is that encryption is part of the transport.

WebRTC data channels use secure transport mechanisms, so you are not sending arbitrary plaintext packets across the internet.

But you still need to think carefully about the broader security model.

For example:

How does someone join a transfer?

Maybe you use:

room code
secret link
QR code

Can someone guess the room?

Your identifiers need enough entropy.

A room system like:

ABC123

is convenient, but the security model depends on:

code length
expiration
rate limiting
session lifetime
whether links contain additional secrets

What does the signaling server know?

Ideally:

connection metadata
temporary room information

but not:

permanent copies of user files

How long do sessions live?

Temporary sessions should actually be temporary.

Security is not just encryption.

It is also minimizing what exists and for how long.

Why I did not want file storage in the architecture

There are obvious infrastructure reasons.

If ButterShare stored every file, I would eventually need to solve:

object storage
storage lifecycle
cleanup jobs
bandwidth costs
abuse
copyright complaints
malware storage
retention policies
privacy
expired transfers
storage quotas

Removing persistent file storage does not eliminate every problem.

But it dramatically changes the product.

The infrastructure becomes much more about:

connection coordination

than:

hosting arbitrary user data

That is a very different operational model.

P2P does not mean free infrastructure

This is worth emphasizing.

People sometimes hear peer-to-peer and assume:

No server costs.

Not true.

You still need infrastructure for things like:

signaling
authentication if applicable
room coordination
STUN
TURN
monitoring
abuse prevention
analytics
APIs

And TURN can become expensive if a meaningful percentage of transfers are relayed.

If a 100GB transfer has to go through your TURN infrastructure, you are moving a lot of bandwidth.

So the business model still needs to understand relay rates.

You should measure things like:

% direct peer connections
% relayed connections
average transfer size
average duration
connection failure rate
reconnect rate

Without this data, you do not really know how your P2P architecture behaves in the real world.

Connection speed is not magic

If two users have slow internet, P2P does not fix physics.

Suppose:

Sender upload: 20 Mbps
Receiver download: 500 Mbps

The transfer is still limited by approximately the sender's available upload bandwidth.

Likewise, Wi-Fi quality, mobile signal, congestion, TURN routing, and device performance can all affect throughput.

“No server upload step” does not mean:

instant 100GB transfer.

100GB is still 100GB.

What changes is where the data goes.

Instead of:

100GB to cloud
then
100GB from cloud

you are trying to make the destination the receiver from the beginning.

Why this architecture becomes attractive for large files

For small files, the benefits may not feel dramatic.

If you are sending a 3MB image, uploading it to a server is basically free from the user's perspective.

But consider:

5GB
20GB
50GB
100GB
500GB

As the file size increases, the architecture matters more.

Cloud storage adds:

an intermediate upload
storage requirements
infrastructure bandwidth
retention management

Direct transfer removes the need for permanent file hosting.

That is why I think P2P becomes increasingly interesting as file size grows.

The browser is becoming an operating system

One of the things I find most interesting about building ButterShare is how much browsers can now do.

A web application can access:

files
directories
WebRTC
streaming APIs
cryptography
persistent storage
service workers
filesystem capabilities on supported platforms

Years ago, a cross-platform file-transfer product almost certainly meant shipping:

Windows app
macOS app
Android app
iOS app
Linux app

Now a surprising amount can be done from:

https://...

That has huge distribution advantages.

The recipient does not necessarily need to:

download an installer
create an account
configure software
update an application

They can open a URL.

That is one reason I think browser-based file transfer is more interesting now than it used to be.

But browser compatibility keeps you humble

Just because an API exists does not mean it behaves identically everywhere.

Anyone building serious browser software learns this quickly.

You have:

Chrome
Edge
Firefox
Safari
iOS Safari
Android Chrome
in-app browsers
older devices

And each one comes with different behavior around:

WebRTC
file APIs
memory
downloads
filesystem writing
background tabs
device sleep
permission prompts

The real product is not:

“It works on my machine.”

The real product is:

“Can an iPhone user send a huge file to a Windows user without understanding WebRTC?”

That is a much harder standard.

UX matters as much as networking

Users do not care about:

ICE candidates
SCTP
DTLS
NAT traversal
bufferedAmount
TURN allocation

They care about:

“Is my file sending?”

“How long will it take?”

“Did it fail?”

“Can I try again?”

“Can I close this tab?”

This means a large part of the engineering work ends up in UX.

For example:

Bad

Peer connection state: disconnected

Better

Connection interrupted. Trying to reconnect...

Bad

bufferedAmount = 4194304

Better

42.7GB of 100GB transferred
Estimated time remaining: 18 minutes

The networking layer can be brilliant, but if the UI makes the user nervous, the product still feels broken.

What I would build differently if I started again

The biggest lesson is that I would treat the transfer engine as its own system from day one.

Not as:

some WebRTC code inside a React component

but as a proper state machine.

Something like:

IDLE
  |
  v
CREATING_SESSION
  |
  v
WAITING_FOR_PEER
  |
  v
CONNECTING
  |
  v
READY
  |
  v
TRANSFERRING
  |
  +----> PAUSED
  |
  +----> RECONNECTING
  |
  +----> FAILED
  |
  v
COMPLETED

That structure makes everything easier:

reconnection
retries
UI
analytics
debugging
resume support

Large file transfer has too many edge cases to manage with a couple of booleans like:

isConnected
isSending

Eventually those booleans turn into chaos.

I would also separate protocol from transport

Another design decision I strongly recommend:

Do not make your application protocol depend too tightly on WebRTC.

Think of WebRTC as the transport.

Your transfer protocol should define concepts such as:

session start
file metadata
chunk
acknowledgement
resume
file complete
transfer complete
error

Then the transport layer handles:

send(message)
receive(message)

That makes the architecture easier to test and potentially easier to extend later.

For example, you could theoretically support another transport in the future without rewriting the entire transfer model.

Measuring the right things

A transfer product needs good telemetry.

Not the files.

Not private user data.

But operational metrics.

I want to know things like:

connection success rate
time to connect
direct vs relayed connections
average throughput
transfer completion rate
disconnect rate
browser distribution
OS distribution
average file size
failure stage

For example, if Safari has:

62% completion

while Chrome has:

94% completion

that tells you exactly where to focus engineering effort.

If transfers larger than:

40GB

fail disproportionately often, that tells you something else.

Without metrics, you are debugging anecdotes.

The most important lesson: 100GB is not just a bigger 1GB

This is probably the main thing I learned.

You cannot build file transfer for 500MB and assume it will naturally scale to 100GB.

At very large sizes, every small weakness becomes visible.

A tiny memory leak matters.

A slightly inaccurate progress bar matters.

A reconnect bug matters.

A 0.5% failure rate matters.

A browser sleeping after 30 minutes matters.

The transfer may be running for hours.

The software has to survive that.

Large-file engineering is less about:

“Can bytes move?”

and more about:

“Can bytes keep moving reliably for a very long time under imperfect real-world conditions?”

That is the actual problem.

Where ButterShare fits

ButterShare is my attempt to make this architecture useful to normal people.

The user should not need to know what WebRTC is.

They should be able to:

Select a file or folder.
Get a link or QR code.
Send it to someone.
Transfer the file directly.

The product is built around:

browser-to-browser transfer
peer-to-peer networking
no traditional cloud file storage
cross-platform usage
large files
no mandatory account creation

The technical complexity should stay underneath.

The user should just see:

Send file.

When P2P is the wrong choice

I do not think peer-to-peer is automatically superior.

There are situations where cloud storage is clearly better.

For example:

Sender needs to go offline

Use cloud storage.

Recipient will download tomorrow

Use cloud storage.

50 people need the same file

Cloud/CDN distribution probably makes more sense.

You need permanent storage

Obviously use storage.

You need versioning and collaboration

Again, cloud storage.

P2P becomes compelling when:

one sender
+
one receiver
+
both online
+
large file
+
no need for permanent hosting

That is the sweet spot.

Final architecture

At a high level, the system looks something like this:

             +----------------------+
             |   Signaling Server   |
             +----------------------+
                  ^             ^
                  |             |
                  | metadata    |
                  | only        |
                  |             |
                  v             v

          +-------------+   +-------------+
          |   Sender    |   |  Receiver   |
          |   Browser   |   |   Browser   |
          +-------------+   +-------------+
                 \             /
                  \           /
                   \         /
                    \       /
                WebRTC Data Channel
                       |
                       |
                Direct if possible
                       |
                       v

               File data moves
             device-to-device

When direct connectivity is impossible:

Sender
  |
  v
TURN relay
  |
  v
Receiver

The important architectural goal remains:

The file does not need to become a permanently stored cloud object just to move between two people.

Closing thoughts

Before building ButterShare, “file sharing” sounded like a solved problem.

Then I started thinking about 100GB files.

The moment files get that large, you discover that the problem is not just uploading.

It is:

transport
memory
backpressure
browser limitations
NAT traversal
mobile lifecycle
reconnection
resume
integrity
filesystem writing
progress reporting
infrastructure economics

And that is what makes the problem interesting.

There is something satisfying about the simplicity of the final idea:

You have the file.

They need the file.

Connect the two devices.

Everything else is engineering.

I'm building this approach into ButterShare, a browser-based large-file transfer tool focused on direct device-to-device transfers without traditional cloud file storage.

If you work with WebRTC, browser streaming, file APIs, or large-file infrastructure, I'd genuinely be interested in hearing how you would approach the same problem.

ButterShare:

https://buttershare.com/

TL;DR

For very large browser-based transfers:

Don't load the entire file into memory.
Break files into manageable chunks.
Respect WebRTC backpressure.
Keep signaling separate from actual file transfer.
Use STUN and TURN appropriately.
Expect mobile browsers to behave differently.
Design reconnection and resume early.
Do not accumulate the complete file in receiver memory.
Track confirmed progress, not just queued bytes.
Treat file transfer as a protocol, not a loop calling send().
Remember that P2P does not mean servers disappear.
Most importantly: 100GB is not simply a larger version of 1GB.

That last point changes almost every engineering decision.

I Built ButterShare: Free P2P File Transfer Without Cloud Storage

Anirban Roy — Tue, 22 Sep 2026 05:57:32 +0000

You have a large video on your phone. You need it on your laptop.

You don’t need a shared drive, another account, or a permanently stored copy. You just need the file on the other device.

That’s the problem I’m working on with ButterShare: a free, browser-based file transfer tool built around peer-to-peer sharing.

I’m the developer behind it. Here’s what it does, why I chose this approach, and the tradeoffs worth understanding.

What is ButterShare?

ButterShare lets you send files between devices through a web browser, without signing up or installing an app.

The main features are:

Browser-to-browser file transfers using WebRTC.
No product-imposed file-size cap.
No cloud storage of your transferred files.
Sharing through a link or QR code.
No compression of the files you send.
Support for cross-platform workflows using compatible browsers.

The important distinction: this is a live transfer, not an upload-and-download-later service. Both devices need to stay connected until the transfer finishes.

How to transfer files with ButterShare

The workflow is straightforward:

Open ButterShare on the sending device.
Select the files you want to share.
Share the generated link or let the recipient scan the QR code.
Open the link on the receiving device and connect.
Keep both browser tabs open until the transfer completes.

You don’t need to create an account for a one-time transfer.

This is useful when moving photos to another device, sharing a screen recording, or sending an exported video to someone who is available to receive it.

Why use WebRTC for file transfer?

WebRTC is often associated with video calls, but its data channels can carry binary data too.

An RTCDataChannel lets browsers exchange data over a peer connection. Data channels use DTLS encryption, and their APIs expose buffering information that applications can use to manage outgoing data.

For developers exploring the underlying technology, MDN’s guide to WebRTC data channels is a useful starting point.

Large-file transfers also involve practical engineering decisions: splitting files into manageable chunks, controlling how much data is queued, and handling the receiving browser’s storage and memory constraints.

That’s why “no product-imposed file-size cap” shouldn’t be confused with “every browser can handle an arbitrarily large file under any conditions.”

No cloud storage does not mean no infrastructure

These are different claims.

A browser-to-browser application can still need infrastructure to help devices discover and connect to each other. Depending on a WebRTC application's configuration and network conditions, connectivity may also involve relay servers.

Temporarily relaying encrypted traffic is different from storing a file for someone to download later.

ButterShare’s file-sharing model is about live delivery without maintaining a cloud-stored copy—not about pretending that networking infrastructure disappears.

When browser-based file sharing makes sense

Moving files between different operating systems

If you’re looking for a browser-based alternative to an AirDrop-style workflow, ButterShare is designed for sharing across devices such as Windows PCs, Macs, Android phones, iPhones, and iPads.

It does not use Apple’s AirDrop protocol. Both devices connect through their browsers instead.

Sending videos without compression

Sometimes you need to send the original exported video, not a smaller version.

ButterShare transfers the selected file without deliberately re-encoding it. That makes it useful when preserving the original file matters.

Sharing files without creating accounts

A quick handoff shouldn’t always require inviting someone into a shared workspace.

A link-based session is useful when both people are already online and just need to exchange files.

The limitations are important

Peer-to-peer transfer is not the right answer for every situation.

Keep these constraints in mind:

Both devices must remain connected. Closing a tab, putting a device to sleep, or losing connectivity can interrupt the transfer.
Speed depends on the connection. Your network, the recipient’s network, and the connection path all matter.
Browser behavior matters. Mobile background restrictions and available memory or storage can affect large transfers.
There is no stored copy waiting for an offline recipient. If someone needs to download tomorrow after you disconnect, a cloud-based transfer service is a better fit.

I don’t want to market this as a replacement for every file-sharing workflow. It solves a specific one: moving files between people or devices that can connect at the same time.

Try it and tell me what breaks

ButterShare’s live file-transfer tool is free to use.

If you try it, I’d appreciate concrete feedback:

Which devices and browsers did you use?
What size and type of file did you transfer?
Did the connection work on the first attempt?
Was anything confusing about sending or receiving?

Please don’t include private filenames or sharing links in your feedback.

Try ButterShare →

What’s the most frustrating part of transferring files between your devices?

5 Tools for Developers to Grow Their Social Presence

Anirban Roy — Mon, 12 Feb 2024 05:11:22 +0000

Having the appropriate too
ls at your disposal may make all the difference in the fast-paced world of social media marketing when it comes to expanding your online presence, interacting with your audience, and producing results. Utilizing the potential of these five vital tools will help you advance your social media management career, regardless of experience level.

1. Canva

Having the appropriate tools at your disposal may make all the difference in the fast-paced world of social media marketing when it comes to expanding your online presence, interacting with your audience, and producing results. Utilizing the potential of these five vital tools will help you advance your social media management career, regardless of experience level.

2. Buffer

A social media management application called Buffer makes it easier to plan, publish, and analyze your content on several social media networks. You can maximize your reach and maintain a consistent posting schedule by planning and scheduling your content ahead of time with Buffer. You may optimize your social media strategy by tracking the effectiveness of your postings, analyzing important indicators, and making data-driven decisions using its user-friendly dashboard.

3. Hootsuite

Another well-known social media management tool is Hootsuite, which provides an extensive toolkit for planning, keeping track of, and evaluating your social media activities. With Hootsuite, you can automate scheduling to optimize your workflow, monitor discussions and mentions in real-time, and manage various social media accounts from a single dashboard. Regardless of your role in a larger marketing team or as a solopreneur, Hootsuite gives you the tools you need to remain productive, efficient, and organized on social media.

4. Moiful

For developers and designers who want to quickly and easily make beautiful screenshots and social media photographs, Moiful is an essential tool. Moiful's user-friendly interface and adaptable template options let you present your projects, application interfaces, or code snippets in an eye-catching manner. When it comes to promoting a blog post, revealing a new feature, or reporting a development milestone, Moiful assists you in producing eye-catching graphics that will grab the attention of your audience and improve your online visibility.

5. BuzzSumo

BuzzSumo is a potent tool for social media analytics and content discovery that enables you to track hot topics, evaluate the performance of competitors, and find information to guide your content strategy. BuzzSumo allows you to track the performance of your content in real-time, keep an eye on mentions of your brand or keywords, and find the most shared material across social media platforms. Using BuzzSumo's data-driven insights, you can expand your audience, provide content that is more engaging, and encourage genuine social media interaction.

Conclusion

Having the appropriate tools is crucial for expanding your social media presence, interacting with your audience, and producing results in the cutthroat world of today. These five indispensable tools—Canva, Buffer, Hootsuite, Moiful, and BuzzSumo—can benefit solopreneurs, small business owners, and members of bigger marketing teams alike by streamlining workflows, maximizing strategies, and helping you accomplish social media objectives. Add these resources to your arsenal right now, and you'll see a dramatic increase in your social media presence.