[whispersystems] Reliable transport/delivery for TextSecure

[Robert Buchholz]

Hello all,

TextSecure currently does not handle message delivery reliably. I
understand this is a known problem, and the introduction of delivery
receipt helps a lot as senders can re-send messages they feel should
have been delivered.

Unfortunately, failure to deliver messages is not that rare. There are
several ([1], [2], [3]) reports from people mentioning random missing
messages. In my own circles and communications, I would say that about
5% of messages are not delivered.

I completely understand the motivation behind not storing the message
and just handing it off to GCM. However, I was wondering whether there
has been a discussion about this issue before and which solutions have
been discussed?

Cheers,

Robert

[1]
http://support.whispersystems.org/customer/portal/questions/8542401-where-did-my-textsecure-messages-go-for-3-hours-2-still-waiting-for-answer-
[2]
https://github.com/WhisperSystems/TextSecure/issues/2108#issuecomment-64766438
[3] https://github.com/WhisperSystems/TextSecure/issues/1971

[Moxie Marlinspike]

So far all the instances of message loss that we've encountered have all
been cases of people doing weird shit to their phones, network
interference, devices without enough disk space to receive the pushes,
etc. We could keep the messages in a queue on the server for delivery,
but I think those problematic devices would just have the same problems.
Presumably GCM has an identical queue somewhere.

We might do it just to be symmetric with iOS, but I doubt reliability
will increase for those people.

- moxie

Post by Robert Buchholz
Hello all,
TextSecure currently does not handle message delivery reliably. I
understand this is a known problem, and the introduction of delivery
receipt helps a lot as senders can re-send messages they feel should
have been delivered.
Unfortunately, failure to deliver messages is not that rare. There are
several ([1], [2], [3]) reports from people mentioning random missing
messages. In my own circles and communications, I would say that about
5% of messages are not delivered.
I completely understand the motivation behind not storing the message
and just handing it off to GCM. However, I was wondering whether there
has been a discussion about this issue before and which solutions have
been discussed?
Cheers,
Robert
[1]
http://support.whispersystems.org/customer/portal/questions/8542401-where-did-my-textsecure-messages-go-for-3-hours-2-still-waiting-for-answer-
[2]
https://github.com/WhisperSystems/TextSecure/issues/2108#issuecomment-64766438
[3] https://github.com/WhisperSystems/TextSecure/issues/1971

[Sam Lanning]

I still think there is definite room for improving UI hints for missing /
out of order messages, given we have all the data to assert these facts...

Cheers,
Sam.

Post by Moxie Marlinspike
So far all the instances of message loss that we've encountered have all
been cases of people doing weird shit to their phones, network
interference, devices without enough disk space to receive the pushes,
etc. We could keep the messages in a queue on the server for delivery,
but I think those problematic devices would just have the same problems.
Presumably GCM has an identical queue somewhere.
We might do it just to be symmetric with iOS, but I doubt reliability
will increase for those people.
- moxie

Post by Robert Buchholz
Hello all,
TextSecure currently does not handle message delivery reliably. I
understand this is a known problem, and the introduction of delivery
receipt helps a lot as senders can re-send messages they feel should
have been delivered.
Unfortunately, failure to deliver messages is not that rare. There are
several ([1], [2], [3]) reports from people mentioning random missing
messages. In my own circles and communications, I would say that about
5% of messages are not delivered.
I completely understand the motivation behind not storing the message
and just handing it off to GCM. However, I was wondering whether there
has been a discussion about this issue before and which solutions have
been discussed?
Cheers,
Robert
[1]
http://support.whispersystems.org/customer/portal/questions/

8542401-where-did-my-textsecure-messages-go-for-3-
hours-2-still-waiting-for-answer-

Post by Robert Buchholz
[2]
https://github.com/WhisperSystems/TextSecure/issues/2108#issuecomment-

64766438

Post by Robert Buchholz
[3] https://github.com/WhisperSystems/TextSecure/issues/1971

--
http://www.thoughtcrime.org

[Martin Lablans]

Hi all,

Post by Moxie Marlinspike
So far all the instances of message loss that we've encountered have all
been cases of people doing weird shit to their phones, network
interference, devices without enough disk space to receive the pushes,
etc.

I'm afraid that's not correct. My whole family is experiencing frequent
message losses using Nexus 4 and 5 phones, standard Google images (4.4 as
well as 5.0). Network interference is, of course, possible. However, a
reliable messager should be able to handle that - driving through a tunnel
should not cause losing messages.

GCM is known to sometimes lose messages. That's okay, so does UDP. However,
implementations relying on that transport need to take into account that
unreliability. The TextSecure server must not "fire and forget" messages
but needs to make sure they are actually received. Thanks to the recent
implementation of delivery receipts, this should now be easier than ever.

There are a lot of possible fixes. The easiest seems to me a server-side
queue: Keep the message on the server. Re-send every n minutes (e.g. n=5).
Delete once receipt has been received.

Con:
- Phones may receive messages multiple times (e.g. when returning from
airplane mode) ---> on the phone, discard messages that have already been
received.
- Server may become cluttered with messages (e.g. to users who have
uninstalled TextSecure) ---> flush a queue if the client has not reported
in for m days (e.g. m=60).

As always, thanks for considering and all your good work.

Cheers
Martin

[Eric Mill]

I'm not personally experiencing lost messages (that I'm aware of), but this
seems like a smart move, especially as TextSecure broadens its appeal to a
larger audience that I suspect is much less tolerant of any lost messages
at all.

Having some in-TextSecure redelivery logic would also make the system less
dependent on any particular 3rd-party delivery mechanism (e.g. GCM vs
WebSockets), and would give TextSecure some helpful analytics on
deliverability.

-- Eric

Post by Martin Lablans
Hi all,

Post by Moxie Marlinspike
So far all the instances of message loss that we've encountered have all
been cases of people doing weird shit to their phones, network
interference, devices without enough disk space to receive the pushes,
etc.

I'm afraid that's not correct. My whole family is experiencing frequent
message losses using Nexus 4 and 5 phones, standard Google images (4.4 as
well as 5.0). Network interference is, of course, possible. However, a
reliable messager should be able to handle that - driving through a tunnel
should not cause losing messages.
GCM is known to sometimes lose messages. That's okay, so does UDP.
However, implementations relying on that transport need to take into
account that unreliability. The TextSecure server must not "fire and
forget" messages but needs to make sure they are actually received. Thanks
to the recent implementation of delivery receipts, this should now be
easier than ever.
There are a lot of possible fixes. The easiest seems to me a server-side
queue: Keep the message on the server. Re-send every n minutes (e.g. n=5).
Delete once receipt has been received.
- Phones may receive messages multiple times (e.g. when returning from
airplane mode) ---> on the phone, discard messages that have already been
received.
- Server may become cluttered with messages (e.g. to users who have
uninstalled TextSecure) ---> flush a queue if the client has not reported
in for m days (e.g. m=60).
As always, thanks for considering and all your good work.
Cheers
Martin

[Corvin Russell]

I would agree and also say that so far TextSecure is the only app I have
experienced GCM-related issues like dropped messages with, so it doesn't
make sense to me to blame network interruptions, etc. - surely if these
were the effective cause then other apps would also be similarly afflicted.

Post by Martin Lablans
Hi all,

Post by Moxie Marlinspike
So far all the instances of message loss that we've encountered have all
been cases of people doing weird shit to their phones, network
interference, devices without enough disk space to receive the pushes,
etc.

I'm afraid that's not correct. My whole family is experiencing frequent
message losses using Nexus 4 and 5 phones, standard Google images (4.4 as
well as 5.0). Network interference is, of course, possible. However, a
reliable messager should be able to handle that - driving through a tunnel
should not cause losing messages.
GCM is known to sometimes lose messages. That's okay, so does UDP.
However, implementations relying on that transport need to take into
account that unreliability. The TextSecure server must not "fire and
forget" messages but needs to make sure they are actually received. Thanks
to the recent implementation of delivery receipts, this should now be
easier than ever.
There are a lot of possible fixes. The easiest seems to me a server-side
queue: Keep the message on the server. Re-send every n minutes (e.g. n=5).
Delete once receipt has been received.
- Phones may receive messages multiple times (e.g. when returning from
airplane mode) ---> on the phone, discard messages that have already been
received.
- Server may become cluttered with messages (e.g. to users who have
uninstalled TextSecure) ---> flush a queue if the client has not reported
in for m days (e.g. m=60).
As always, thanks for considering and all your good work.
Cheers
Martin

--
Twitter: @corvinr <https://twitter.com/corvinr>
Google Plus: https://plus.google.com/115459943980453987525
My PGP key is here
<http://pgp.mit.edu:11371/pks/lookup?op=get&search=0x48CBE0180B9915EF>.

Help restore privacy and freedom of association for everyone by using Signal
<https://itunes.apple.com/app/id874139669> on iPhone and TextSecure
<https://play.google.com/store/apps/details?id=org.thoughtcrime.securesms&hl=en>
and RedPhone
<https://play.google.com/store/apps/details?id=org.thoughtcrime.redphone&hl=en>
on
Android.

[Robert Buchholz]

Hello all,

Post by Martin Lablans

Post by Moxie Marlinspike
So far all the instances of message loss that we've encountered have all
been cases of people doing weird shit to their phones, network
interference, devices without enough disk space to receive the pushes,
etc.

I'm afraid that's not correct. My whole family is experiencing frequent
message losses using Nexus 4 and 5 phones, standard Google images (4.4 as
well as 5.0). Network interference is, of course, possible. However, a
reliable messager should be able to handle that - driving through a tunnel
should not cause losing messages.

I fully have to agree. I have quite a few people on TextSecure, and loss
of messages is rather common. While it rarely happens when both parties
are in a home/office wireless, I have actually encountered loss of a
single message within a successful conversation of a few minutes.

The bigger point is what Moxie called "network interference", because at
least in Germany, where I live, mobile data service is unreliable.
Connection loss is common in parts of the subway and on speed trains, or
when changing from wireless to mobile connection. Those are times when I
often use TextSecure to coordinate with others.

I would like to raise this issue as not being limited to people modding
their devices, but rather happening to almost all stock Android 2 and 4
users I have had discussions with. So let's at least have the conversion
what is needed to get this fixed.


Cheers,

Robert

[Steffen Märcker]

I can only second this. Lost messages are pretty annoying and almost all
of my friends encounter this issue regularly. From my observation it
happens if the receiver has an unstable data connection or a significant
network latency. I face this problem each month when my data plan is cut
down to snail-net after consuming a GB of data.

A reasonable solution I can think of would a client-side resend mechanism
for the sender together with a dedupe on the receivers side.

Best, Steffen

Am .12.2014, 16:20 Uhr, schrieb Robert Buchholz

Post by Robert Buchholz
Hello all,

Post by Martin Lablans

Post by Moxie Marlinspike
So far all the instances of message loss that we've encountered have

all

Post by Moxie Marlinspike
been cases of people doing weird shit to their phones, network
interference, devices without enough disk space to receive the pushes,
etc.

I'm afraid that's not correct. My whole family is experiencing frequent
message losses using Nexus 4 and 5 phones, standard Google images (4.4 as
well as 5.0). Network interference is, of course, possible. However, a
reliable messager should be able to handle that - driving through a tunnel
should not cause losing messages.

I fully have to agree. I have quite a few people on TextSecure, and loss
of messages is rather common. While it rarely happens when both parties
are in a home/office wireless, I have actually encountered loss of a
single message within a successful conversation of a few minutes.
The bigger point is what Moxie called "network interference", because at
least in Germany, where I live, mobile data service is unreliable.
Connection loss is common in parts of the subway and on speed trains, or
when changing from wireless to mobile connection. Those are times when I
often use TextSecure to coordinate with others.
I would like to raise this issue as not being limited to people modding
their devices, but rather happening to almost all stock Android 2 and 4
users I have had discussions with. So let's at least have the conversion
what is needed to get this fixed.
Cheers,
Robert

[[e-mail]]

I encountered message loss some times as well, but I always thought
its my fault switching between several self-compiled versions etc...

I tried to look into that, and modified TS to send 50 numbered
messages when you click the send button:
https://github.com/agrajaghh/TextSecure/tree/batch_sending_test

I'm running this branch in an emulator sending messages to my stock
Nexus 4 Android 5 Rom (nothing weird) with a clean master install
(uninstalled TS before). Sending these 50 messages takes ofc some
time. I switched off my Nexus 4 while the emulator was still sending.
After switching the phone on again, I didn't receive any further
messages. It seems they are really lost! I was also able to reproduce
this, so I think thats really an issue...

Do you have any idea why GCM doesn't send them when the phone is
switched on again?

best regards

Post by Moxie Marlinspike
So far all the instances of message loss that we've encountered have all
been cases of people doing weird shit to their phones, network
interference, devices without enough disk space to receive the pushes,
etc. We could keep the messages in a queue on the server for delivery,
but I think those problematic devices would just have the same problems.
Presumably GCM has an identical queue somewhere.
We might do it just to be symmetric with iOS, but I doubt reliability
will increase for those people.
- moxie

Post by Robert Buchholz
Hello all,
TextSecure currently does not handle message delivery reliably. I
understand this is a known problem, and the introduction of delivery
receipt helps a lot as senders can re-send messages they feel should
have been delivered.
Unfortunately, failure to deliver messages is not that rare. There are
several ([1], [2], [3]) reports from people mentioning random missing
messages. In my own circles and communications, I would say that about
5% of messages are not delivered.
I completely understand the motivation behind not storing the message
and just handing it off to GCM. However, I was wondering whether there
has been a discussion about this issue before and which solutions have
been discussed?
Cheers,
Robert
[1]
http://support.whispersystems.org/customer/portal/questions/8542401-where-did-my-textsecure-messages-go-for-3-hours-2-still-waiting-for-answer-
[2]
https://github.com/WhisperSystems/TextSecure/issues/2108#issuecomment-64766438
[3] https://github.com/WhisperSystems/TextSecure/issues/1971

--
http://www.thoughtcrime.org

[Steffen Märcker]

Do you have any update for us on this issue?

Best, Steffen

Post by Moxie Marlinspike
So far all the instances of message loss that we've encountered have all
been cases of people doing weird shit to their phones, network
interference, devices without enough disk space to receive the pushes,
etc. We could keep the messages in a queue on the server for delivery,
but I think those problematic devices would just have the same problems.
Presumably GCM has an identical queue somewhere.
We might do it just to be symmetric with iOS, but I doubt reliability
will increase for those people.
- moxie

Post by Robert Buchholz
Hello all,
TextSecure currently does not handle message delivery reliably. I
understand this is a known problem, and the introduction of delivery
receipt helps a lot as senders can re-send messages they feel should
have been delivered.
Unfortunately, failure to deliver messages is not that rare. There are
several ([1], [2], [3]) reports from people mentioning random missing
messages. In my own circles and communications, I would say that about
5% of messages are not delivered.
I completely understand the motivation behind not storing the message
and just handing it off to GCM. However, I was wondering whether there
has been a discussion about this issue before and which solutions have
been discussed?
Cheers,
Robert
[1]
http://support.whispersystems.org/customer/portal/questions/8542401-where-did-my-textsecure-messages-go-for-3-hours-2-still-waiting-for-answer-
[2]
https://github.com/WhisperSystems/TextSecure/issues/2108#issuecomment-64766438
[3] https://github.com/WhisperSystems/TextSecure/issues/1971

[Eric Mill]

Consider opening a ticket in the GitHub project.

Post by Steffen Märcker
Do you have any update for us on this issue?
Best, Steffen

Post by Moxie Marlinspike
So far all the instances of message loss that we've encountered have all
been cases of people doing weird shit to their phones, network
interference, devices without enough disk space to receive the pushes,
etc. We could keep the messages in a queue on the server for delivery,
but I think those problematic devices would just have the same problems.
Presumably GCM has an identical queue somewhere.
We might do it just to be symmetric with iOS, but I doubt reliability
will increase for those people.
- moxie

Post by Robert Buchholz
Hello all,
TextSecure currently does not handle message delivery reliably. I
understand this is a known problem, and the introduction of delivery
receipt helps a lot as senders can re-send messages they feel should
have been delivered.
Unfortunately, failure to deliver messages is not that rare. There are
several ([1], [2], [3]) reports from people mentioning random missing
messages. In my own circles and communications, I would say that about
5% of messages are not delivered.
I completely understand the motivation behind not storing the message
and just handing it off to GCM. However, I was wondering whether there
has been a discussion about this issue before and which solutions have
been discussed?
Cheers,
Robert
[1]
http://support.whispersystems.org/customer/portal/questions/
8542401-where-did-my-textsecure-messages-go-for-3-
hours-2-still-waiting-for-answer-
[2]
https://github.com/WhisperSystems/TextSecure/issues/2108#issuecomment-
64766438
[3] https://github.com/WhisperSystems/TextSecure/issues/1971

[Steffen Märcker]

I thought #1971 covers the problem well, doesn't it?

Best, Steffen

Post by Eric Mill
Consider opening a ticket in the GitHub project.

Post by Steffen Märcker
Do you have any update for us on this issue?
Best, Steffen
Am .11.2014, 16:10 Uhr, schrieb Moxie Marlinspike

<---Schnitt--->

[Robert Buchholz]

Hello,

Post by Robert Buchholz
Unfortunately, failure to deliver messages is not that rare. There are
several ([1], [2], [3]) reports from people mentioning random missing
messages. In my own circles and communications, I would say that about
5% of messages are not delivered.

I'm not sure how closely you follow the related issues on GitHub, but
there's continuous feedback from people reporting on reliability
issues [1]. Is this list the place to discuss whether this problem
should be addressed and how?


Cheers,

Robert


[1] GitHub issues:
https://github.com/WhisperSystems/TextSecure/issues/1971
https://github.com/WhisperSystems/TextSecure/issues/970
https://github.com/WhisperSystems/TextSecure/issues/2098
https://github.com/WhisperSystems/TextSecure/issues/2238