Sending our public key in header of message
Now attaching our key to the source field when sending messages, allows messages to be decrypted with the fallback cypher
Now polling the server for messages every 5 seconds
Sending the source device with messages
Added mock respond function to request to leave it that same as the websocket stuff. RetrieveMessages now just returns the result
Polling now continues if the server responds with an error. Returning only the result from sendMessage and retrieveMessages
Revert commenting of unreachable code
Refactored http logic into own file
Revert a change to websocket-resources
* 'master' of https://github.com/signalapp/Signal-Desktop: (38 commits)
v1.17.0
v1.17.0-beta.4
Ensure that blue group avatars are preserved in dark theme
Android theme: Incoming quotes take color from containing message
Conversations have blue avatar backgrounds if no image provided
Fix lint errors
Add ca language
inboxView: Ensure Conversation exists for our own number
Ensure that file is not attached if we've filtered it
v1.17.0-beta.3
Localization updates
v1.16.3
Lint fixes
Lint fixes
Better handle large numbers of messages in cache on startup
Keep object stores after conversations migrate to SQLCipher
Longer timeout for orphaned file cleanup
Tests and increase consistency for isFileDangerous calls
Longer timeout for orphaned file cleanup
v1.17.0-beta.2
...
# Conflicts:
# background.html
# package.json
Quite a few other fixes, including:
- Sending to contact with no avatar yet (not synced from mobile)
- Left pane doesn't update quickly or at all on new message
- Left pane doesn't show sent or error status
Also:
- Contributing.md: Ensure set of linux dev dependencies is complete
1. MessageReceiver always pulls down thumbnails included in quotes
2. Message.upgradeSchema has a new schema that puts all thumbnails on
disk just like happens with full attachments.
3. handleDataMessage pipes quote from dataMessage into the final message
destined for the database
Synchronizes our protocol buffers with `libsignal-service-java` project.
**Changes**
- [x] **BREAKING:** Rename `package` from `textsecure` to `signalservice`.
⚠️~~Workaround: Rename back to `textsecure`.~~
Changed all protobuf `package` names across our project.
- [x] Rename `java_` metadata.
- [x] Move `NullMessage`, `ReceiptMessage`, and `Verified`.
- [x] Rename `Contacts.isComplete` to `Contacts.complete`. Confirmed to be
unreferenced in our project.
- [x] Rename `Settings` to `Configuration` (`textsecure.protobuf.Settings` seems
to be unused)
- [x] Rename `libtextsecure` `MessageReceiver` `settings` event to
`configuration`.
- [x] Rename `ReceiptMessage.timestamps` to `ReceiptMessage.timestamp`.
- [x] Add `AttachmentPointer` `width` and `height`.
- [x] Renamed `IncomingPushMessageSignal.proto` to `SignalService.proto` to
match server.
---
commit 2b6aa19bf9ea5d8f2f4fd9e4102699a9d06a2b45
Author: Daniel Gasienica <daniel@gasienica.ch>
Date: Wed Feb 14 19:41:24 2018 -0500
Rename protobuf `package`: `textsecure` --> `signalservice`
Brings us closer to `libsignal-service-java`.
commit 91612880a5bf2c9ae8a9334877ac24e91102b905
Author: Daniel Gasienica <daniel@gasienica.ch>
Date: Wed Feb 14 19:19:35 2018 -0500
Rename `SyncMessage.Settings` to `SyncMessage.Configuration`
commit 848eb9559928c54dffd3426bba8e7cd7b1687cdc
Author: Daniel Gasienica <daniel@gasienica.ch>
Date: Tue Feb 13 20:16:43 2018 -0500
Rename `ReadReceipt` `timestamps` --> `timestamp`
commit 39859e64b41ddf41127b52d963fe7cc2b9fcad68
Author: Daniel Gasienica <daniel@gasienica.ch>
Date: Wed Feb 14 18:43:42 2018 -0500
Rename `IncomingPushMessageSignal.proto` to `SignalService.proto`
This matches the `libsignal-service-java` filename.
commit fd4bfd76af57ffa44178caf21d350cca211dc048
Author: Daniel Gasienica <daniel@gasienica.ch>
Date: Tue Feb 13 16:24:36 2018 -0500
Revert protobuf `package` to `textsecure`
This was a breaking change and would need to be introduced with additional
changes.
commit 9f618fa91717a0349f1ea28bf6d365ef0a5c9ca5
Author: Daniel Gasienica <daniel@gasienica.ch>
Date: Tue Feb 13 16:09:55 2018 -0500
Sync service protocol buffers with Java project
Snapshot: 4684a49b2e/protobuf/SignalService.proto
* Log SyncMessage destination, even if it's a group
* Remove disappearing message even if deleted during fetch
This change fixes a potential race condition with disappearing message
removal from the UI. If the UI is in the middle of making a database
request for messages to display, then we need to wait for that to
complete before removing the newly-deleted expired message.
* Log when we get a blocked numbers sync message
* Save three old signed keys in addition to the current active
* Remove the mystery from all the error-related log messages
* Log successful load of signed key - to help debug prekey errors
* removeSignedPreKey: Don't hang or crash in error cases
* Log on top-level unhandled promise rejection
* Remove trailing comma in param list, Electron 1.6 does not like
* Harden top-level error handler for strange object shapes
* web requests: Don't fail if JSON not well-formed in error cases
Turns out that before our move to node-fetch, we ignored JSON.parse()
failures: https://github.com/WhisperSystems/Signal-Desktop/pull/1552/files#diff-1103a6aff0f28b6066715c6994278767L37518
* Update to libsignal v1.2.0, delete sessions on reset session
Previously we only archived sessions when sending or receiving a
'reset secure session, which didn't match up with the mobile apps.
* Online/offline: Always stop timer, don't connect if closed
* Sockets: Send our own close event faster, shutdown only on close
Seems that we were too-aggressively disconnecting from all socket
events. Also, we should be able to send our own close event a lot
faster with no ill effects.
* Catch-up libtextsecure changes
* Ensure that our preload.js setImmediate call finds right function
FREEBIE
* Our own socket close event, better logging, unregistration
FREEBIE
* Return CLOSED for NetworkStatusView if we've fully disconnected
* background.js: Remove messageReceiver = null, log in connect()
A null messageReciever makes the NetworkStatusView think we're online.
FREEBIE
* Refactor delivery receipt event handler
* Rename the delivery receipt event
For less ambiguity with read receipts.
* Rename synced read event
For less ambiguity with read receipts from other Signal users.
* Add support for incoming receipt messages
Handle ReceiptMessages, which may include encrypted delivery receipts or read
receipts from recipients of our sent messages.
// FREEBIE
* Rename ReadReceipts to ReadSyncs
* Render read messages with blue double checks
* Send read receipts to senders of incoming messages
// FREEBIE
* Move ReadSyncs to their own file
// FREEBIE
* Fixup old comments on read receipts (now read syncs)
And some variable renaming for extra clarity.
// FREEBIE
* Add global setting for read receipts
Don't send read receipt messages unless the setting is enabled.
Don't process read receipts if the setting is disabled.
// FREEBIE
* Sync read receipt setting from mobile
Toggling this setting on your mobile device should sync it to Desktop. When
linking, use the setting in the provisioning message.
// FREEBIE
* Send receipt messages silently
Avoid generating phantom messages on ios
// FREEBIE
* Save recipients on the outgoing message models
For accurate tracking and display of sent/delivered/read state, even if group
membership changes later.
// FREEBIE
* Fix conversation type in profile key update handling
// FREEBIE
* Set recipients on synced sent messages
* Render saved recipients in message detail if available
For older messages, where we did not save the intended set of recipients at the
time of sending, fall back to the current group membership.
// FREEBIE
* Record who has been successfully sent to
// FREEBIE
* Record who a message has been delivered to
* Invert the not-clickable class
* Fix readReceipt setting sync when linking
* Render per recipient sent/delivered/read status
In the message detail view for outgoing messages, render each recipient's
individual sent/delivered/read status with respect to this message, as long as
there are no errors associated with the recipient (ie, safety number changes,
user not registered, etc...) since the error icon is displayed in that case.
*Messages sent before this change may not have per-recipient status lists
and will simply show no status icon.
// FREEBIE
* Add configuration sync request
Send these requests in a one-off fashion when:
1. We have just setup from a chrome app import
2. We have just upgraded to read-receipt support
// FREEBIE
* Expose sendRequestConfigurationSyncMessage
// FREEBIE
* Fix handling of incoming delivery receipts - union with array
FREEBIE
* Disconnect from socket faster on complete loss of network access
Today we wait for a keepalive request to fail; this change forces
disconnect in the case that the browser tells us that we're now offline.
FREEBIE
* MessageReceiver: don't react to errors after explicit close()
FREEBIE
Rather than throw an error, just log call messages and drop them. This way we
distinguish them from incorrectly encoded content messages or new types of
messages we don't support yet, and don't insert unnecessary red flags and
stacktraces in debug logs.
// FREEBIE
* Add AES-GCM encryption for profiles
With tests.
* Add profileKey to DataMessage protobuf
// FREEBIE
* Decrypt and save profile names
// FREEBIE
* Save incoming profile keys
* Move pad/unpad to crypto module
// FREEBIE
* Support fetching avatars from the cdn
// FREEBIE
* Translate failed authentication errors
When AES-GCM authentication fails, webcrypto returns a very generic error. The
same error is thrown for invalid length inputs, but our earlier checks in
decryptProfile should rule out those failure modes and leave us safe to assume
that we either had bad ciphertext or the wrong key.
// FREEBIE
* Handle profile avatars (wip) and log decrypt errors
// FREEBIE
* Display profile avatars
Synced contact avatars will still override profile avatars.
* Display profile names in convo list
Only if we don't have a synced contact name.
// FREEBIE
* Make cdn url an environment config
Use different ones for staging and production
// FREEBIE
* Display profile name in conversation header
* Display profile name in group messages
* Update conversation header if profile avatar changes
// FREEBIE
* Style profile names small with ~
* Save profileKeys from contact sync messages
// FREEBIE
* Save profile keys from provisioning messages
For standalone accounts, generate a random profile key.
// FREEBIE
* Special case for one-time sync of our profile key
Android will use a contact sync message to sync a profile key from Android
clients who have just upgraded and generated their profile key. Normally we
should receive this data in a provisioning message.
// FREEBIE
* Infer profile sharing from synced data messages
* Populate profile keys on outgoing messages
Requires that `profileSharing` be set on the conversation.
// FREEBIE
* Support for the profile key update flag
When receiving a message with this flag, don't init a message record, just
process the profile key and move on.
// FREEBIE
* Display profile names in group member list
* Refresh contact's profile on profile key changes
// FREEBIE
* Catch errors on profile save
// FREEBIE
* Save our own synced contact info
Don't return early if we get a contact sync for our own number
// FREEBIE
* Add certificate pinning on https service requests
Make https requests to the server using node apis instead of browser apis, so we
can specify our own CA list, which contains only our own CA.
This protects us from MITM by a rogue CA.
As a bonus, this let's us drop the use of non-standard ports and just use good
ol' default 443 all the time, at least for http requests.
// FREEBIE
* Make certificateAuthorities an option on requests
Modify node-based xhr implementation based on driverdan/node-XMLHttpRequest,
adding support for setting certificate authorities on each request.
This allows us to pin our master CA for requests to the server and cdn but not
to the s3 attachment server, for instance. Also fix an exception when sending
binary data in a request: it is submitted as an array buffer, and must be
converted to a node Buffer since we are now using a node based request api.
// FREEBIE
* Import node-based xhr implementation
Add a copy of https://github.com/driverdan/node-XMLHttpRequest@86ff70e, and
expose it to the renderer in the preload script.
In later commits this module will be extended to support custom certificate
authorities.
// FREEBIE
* Support "arraybuffer" responseType on requests
When fetching attachments, we want the result as binary data rather than a utf8
string. This lets our node-based XMLHttpRequest honor the responseType property
if it is set on the xhr.
Note that naively using the raw `.buffer` from a node Buffer won't work, since
it is a reuseable backing buffer that is often much larger than the actual
content defined by the Buffer's offset and length.
Instead, we'll prepare a return buffer based on the response's content length
header, and incrementally write chunks of data into it as they arrive.
// FREEBIE
* Switch to self-signed server endpoint
* Log more error info on failed requests
With the node-based xhr, relevant error info are stored in statusText and
responseText when a request fails.
// FREEBIE
* Add node-based websocket w/ support for custom CA
// FREEBIE
* Support handling array buffers instead of blobs
Our node-based websocket calls onmessage with an arraybuffer instead of a blob.
For robustness (on the off chance we switch or update the socket implementation
agian) I've kept the machinery for converting blobs to array buffers.
// FREEBIE
* Destroy all wacky server ports
// FREEBIE
On a recent trip through a CPU profile taken while Signal Desktop
churned through a large backlog of messages, it was clear that
console.log was a major source of time spent, primarily the sort
operation required after every new entry is added to the Backbone
collection. So, three different techniques to combat this:
1) Reduce the maximum number of entries in the collection from 5k to 2k
2) No more logging of add/update/remove queue in MessageReceiver
3) No more log entries in Message.handleDataMessage main codepath
FREEBIE
When processing a contact sync with embedded identity key verification info, we
were running overlapping async fetch/save operations on the same conversation
model, causing a race that tends to clobber updates to the contact info.
In this change we extend the application-level contact info handler to block on
a subsequent call to the verification handler, which effectively serializes the
fetch/save calls, and relieves the need for the message receiver to trigger a
seperate event concerning the verification info on contact sync messages.
Fixes#1408
// FREEBIE
* Add support for backup and restore
This first pass works for all stores except messages, pending some scaling
improvements.
// FREEBIE
* Import of messages and attachments
Properly sanitize filenames. Logging information that will help with
debugging but won't threaten privacy (no contact or group names),
where the on-disk directories have this information to make things
human-readable
FREEBIE
* First fully operational single-action export and import!
FREEBIE
* Add migration export flow
A banner alert leads to a blocking ui for the migration. We close the socket and
wait for incoming messages to drain before starting the export.
FREEBIE
* A number of updates for the export flow
1. We don't immediately pop the directory selection dialog box, instead
showing an explicit 'choose directory' button after explaining what is
about to happen
2. We show a 'submit debug log' button on most steps of the process
3. We handle export errors and encourage the user to double-check their
filesystem then submit their log
4. We are resilient to restarts during the process
5. We handle the user cancelling out of the directory selection dialog
differently from other errors.
6. The export process is now serialized: non-messages, then messages.
7. After successful export, show where the data is on disk
FREEBUE
* Put migration behind a flag
FREEBIE
* Shut down websocket before proceeding with export
FREEBIE
* Add MigrationView to test/index.html to fix test
FREEBIE
* Remove 'Submit Debug Log' button when the export process is complete
FREEBIE
* Create a 'Signal Export' directory below user-chosen dir
This cleans things up a bit so we don't litter the user's target
directory with lots of stuff.
FREEBIE
* Clarify MessageReceiver.drain() method comments
FREEBIE
* A couple updates for clarity - event names, else handling
Also the removal of wait(), which wasn't used anywhere.
FREEBIE
* A number of wording updates for the export flow
FREEBIE
* Export complete: put dir on its own line, make text selectable
FREEBIE
There's really no reason to retry encryption errors again if they've
already been made user-visible in a conversation.
Also, refactor e->error in background.js onError(), since both e and ev
in this method made it too easy to make a mistake.
These were failing because ByteBuffers from the protobufs need to be converted
to ArrayBuffers. Fixed by useing the existing handler in MessageReceiver to
process verified messages from contact sync messages and dispatch them as their
own events, reducing some complexity on the application side.
// FREEBIE
Just log em for debugging. Also update the error messge thrown when we get a
content message with no supported properties. It may be empty or may just have
an unrecognized field.
// FREEBIE
This adds a new method to message sender for sending verification sync messages
and a new event to message receiver representing incoming verification sync
messages. Currently the event handler just logs the message.
// FREEBIE
Previously we only expected these errors to occur when decrypting prekey
messages, so we need to add handling for them in the case of a normal message.
// FREEBIE
There may come a day when we may need to change this url from the server
side. On that day, clients should continue to operate normally. The
service should be able to change attachment server locations without
requiring a client update.
// FREEBIE
Because remote clients will delete all sessions in response to an end
session message, regardless of which device it came from, when our
linked device sends an end session message, we must also end all
sessions with the destination.
This change moves the end session flag processing to processDecrypted,
which is shared between handlers of sent messages, data messages, and
messages which are re-tried after resolving identity conflicts.
// FREEBIE
Help debug bad session errors by logging some envelope info about the
message we are about to decrypt. With this, if there is a decryption
error (e.g., bad mac or no session) it is clear from the logs what
number and device message sent the bad message.
Also log when we send and receive end session messages and when we close
sessions for certain devices.
// FREEBIE
Sometimes an error is thrown while processing groups from a group sync
message. We still want to fire the groupsync event when we're done
handling all the data, even if some of it was bad.
We should not ack envelope protobufs that fail to decode correctly. If
the server happens to send us such a thing it probably indicates a
protocol mismatch between it and the client, in which case the client
needs to update and re-receive the failed message.
// FREEBIE
Follow up to b0da4910. When inferring membership of the sender in an
unknown group, remember to save and return the group members.
Generally, this should only effect standalone clients unless someone
managed to clear their groups db table, since linked clients get group
info synced at registration.
// FREEBIE
This may increase processing latency a bit, particularly with large
attachments, but will ensure that messages are dispatched in the order
they are received.
It would be nice to enforce ordering on only the dispatch step, so that
we could, for example, decrypt the next websocket message while waiting
for an attachment to download, but that will require a more complicated
refactor. Will stick with the quick fix for now and revisit later.
Fixes#342
// FREEBIE
Websocket resources should have their keepalive timers reset whenever a
message comes in. This is a nicety that slightly reduces the amount of
traffic we send when actively messaging.
Previously this was handled by MessageReceiver, but it's a bit cleaner
to just have the WebsocketResource add an extra 'message' event handler.
// FREEBIE
This ensures that the containing promise is rejected without triggering
the side effects of an uncaught exception, such as causing the debugger
to pause.
// FREEBIE
`tryMessageAgain` is the routine called when re-trying a message that
failed to decrypt due to an IncomingIdentityKeyError. This handling
needs to move to MessageReceiver because it depends on
`processDecrypted` to handle incoming message protos, which depends
on a server instance in order to download attachments.
// FREEBIE
Following the pattern from previous commit, let the server class accept
a url and login credentials from the caller. Then integrate into
MessageReceiver and AccountManager.
// FREEBIE
Rather than asking for a global target, the message receiver implements
the EventTarget interface itself. It does not expose the dispatchEvent
method, however. This ensures that events can only be triggered from
within the internal MessageReceiver class, which means we no longer need
to namespace them.
// FREEBIE
Let the libtextsecure consumer pass in their own server url, username,
password, and signaling key, as with libtextsecure-java.
Also brings reconnect logic up into the MessageReceiver class, which
is the only place it should apply.
Forgot to bind the socket event handler, and the then() handler should
come before the catch() handler or else it will execute every time the
catch handler executes.
// FREEBIE
Always test connectivity with an http request after a websocket closes,
regardless of what code/error it closed with. If that request succeeds,
automatically reconnect the socket.
// FREEBIE
We now disconnect ourselves if we don't get the server's response to a
keepalive request within 30s. This way we will eventually disconnect if
the network goes away but the socket is not closed.*
* See code.google.com/p/chromium/issues/detail?id=197841 and
https://stackoverflow.com/questions/11755605/chrome-websocket-connection-not-closed-when-browser-closed
We will then try to reconnect once a minute (See 8a10c96);
Keepalives belong at this level anyway, since the format is defined by
both the websocket resource protocol and our specific server url
structure.
// FREEBIE
Protocol and handling is all analogous to contact sync: Multiple
GroupDetails structs are packed into a single attachment blob and parsed
on our end. We don't display the synced groups in the conversation list
until a new message is sent to one of them.
// FREEBIE
Initializing a message receiver opens the socket and starts listening
right away rather than requiring a separate call to connect. The only
other publicly accessible method is to query the socket status.
// FREEBIE
Update protobuf definitions and refactor message receive and decrypt
codepath to support new protocol, including various flavors of sync
messages (sent messages, contacts, and groups).
Also cleans up background.js and lets libtextsecure internalize
textsecure.processDecrypted and ensure that it is called before handing
DataMessages off to the application.
The Envelope structure now has a generic content field and a
legacyMessage field for backwards compatibility. We'll send outgoing
messages as legacy messages, and sync messages as "content" while
continuing to support both legacy and non-legacy messages on the receive
side until old clients have a chance to transition.
Encapsulate the websocket resources and socket setup process in a
friendly OO class. The MessageReceiver constructor expects an instance
of EventTarget on which to fire message events asynchronously. The
provider of the EventTarget can then add/remove listeners as desired.
Mikunj
BeaudanBrown
Scott Nonnenberg
Beaudan
sachaaaaa
sachaaaaa
Daniel Gasienica
Scott Nonnenberg
Lilia
Karel Bilek
Matt Corallo