Storing multiple sessions in a single indexeddb record is prone to
clobbering data due to races between requests to update multiple device
sessions for the same number, since you have to read the current state
of the device->session map and update it. Splitting the records up makes
it so that those updates can be made in parallel. Selecting all the
sessions for a given number can still be done efficiently thanks to
indexeddb range queries.
* Session records are now opaque strings, so treat them that way:
- no more cross checking identity key and session records
- Move hasOpenSession to axolotl wrapper
- Remote registration ids must be fetched async'ly via protocol wrapper
* Implement async AxolotlStore using textsecure.storage
* Add some db stores and move prekeys and signed keys to indexeddb
* Add storage tests
* Rename identityKey storage key from libaxolotl25519KeyidentityKey to
simply identityKey, since it's no longer hardcoded in libaxolotl
* Rework registration and key-generation, keeping logic in libtextsecure
and rendering in options.js.
* Remove key_worker since workers are handled at the libaxolotl level
now
Encapsulate the websocket resources and socket setup process in a
friendly OO class. The MessageReceiver constructor expects an instance
of EventTarget on which to fire message events asynchronously. The
provider of the EventTarget can then add/remove listeners as desired.
Ground work for a smoother registration flow. Overall UX still needs
some polish but at least now we can have a progress gif or animation or
whatever. Also adds the phonenumber-confirmation step as a simple alert
box, which will be replaced with a nice dialogue in a later commit.
Creating a group with a member who's identity key has changed would
previously fail silently. Now, we catch and save the error, allowing the
same conflict resolution process as with regular messages.
Fixes#205
Previously there was a long pause between confirming the group details
and opening the conversation. Fix by first saving/opening the
conversation, rather than waiting for the initial group update to finish
transmitting.
This reverts commit 31e7d285e3.
This seemed like a nice feature, but the popup bubble isn't very
conducive to nontrivial user inputs, e.g. file inputs.
Fixes#211
Clicking on a key conflict message opens the message detail view,
which displays the contact(s) in this conversation. If the message
contains a key conflict with any of these contacts, a button is
displayed which attempts to resolve that conflict and any other
conflicts in the conversation that are related to that contact.
We'd like to live in a world where we can retry all the pending
conflicts in a conversation as a batch, which means we don't want to
wipe the identity key before processing each message. Thus, remove that
step from these handlers and encapsulate in a method on the conversation
model.
Ensure that both tryAgain functions return promises, allowing the
application to take appropriate action in the result of success or
failure. This lets us remove all dependency from libtextsecure on
app-level constructs like message objects/ids and the `extenion.trigger`
function.
Corresponding frontend changes to follow in another commit.
1. Return the value returned by the registered function, to expose the
underlying promise to the caller.
2. Stop accepting extra arguments to the replay function. The caller
should be able to do what they want with the returned promise instead.
3. Add a timestamp argument to the outgoing case, needed to re-try
sending a message.
Rather than opening the inbox in its own window, let it appear as a
browser action popup by default, but allow promotion to its own window
if requested.
Let libaxolotl throw a generic error instead of a replayable error, and
add an helper function in libtextsecure's axolotl_wrapper to catch and
convert from the generic error to the replayable one. This allows the
ReplayableError to remain a libtextsecure-level concept only.
Somewhat unrelatedly, but nearby, fix some whitespace and add missing
semicolon.
`emoji.init_colons` creates and populates `emoji.map.colons`, a global
map from common names to emoji code points. It's safe to call
repeatedly, but unecessary.
Now with actual malloc/free implementations. Had to drop back to -O1
optimization because the ed25519 signature test broke with -O2. :(
Closes#153
The toolchain install and build process, for reference:
```
wget https://s3.amazonaws.com/mozilla-games/emscripten/releases/emsdk-portable.tar.gz
tar xvfz emsdk-portable.tar.gz
cd emsdk-portable
./emsdk update
./emsdk install latest
./emsdk activate latest
source ./emsdk_env.sh
cd ../TextSecure-Browser
grunt build
```
I had some trouble with the registration..
Quickly checked TextSecure Server API and found the problems:
number has to be valid PSTN. (Including regionCode, precisely what validateNumber() returns)
verificationCode has to be all numbers, no dashes or spaces
Closes#193
Update unreadCounts per-conversation on incoming messages. Render unread
conversations with font-weigh: bold in the inbox view.
To ensure that the inbox and conversation views remain in sync, the
background page now ensures that the same models objects are used for
both views.
Turns out we can get ABNORMAL_CODE (1006) for disconnects where (for
instance) we pause the background page too long. However, in these cases
there is no preceeding ErrorEvent. In contrast, when we have bad
authentication credentials, there is an ErrorEvent. Thus, this change
ensures that we only reconnect if there was no Error.
Templatize the inbox view and use the same pattern for in-window view
switching as is now used with the conversation/message detail views.
This means doing more with markup and less jquery manipulation of
individual subelements of the inbox view.
Closes#173
Previously, in the event of a failed websocket auth, we would attempt to
reconnect once a second ad infinitum. This changeset ensures that we
only reconnect automatically if the socket closed 'normally' as
indicated by the code on the socket's CloseEvent. Otherwise, show a
'Websocket closed' error on the inbox view.
Ideally we would show a more contextual error (ie, 'Unauthorized'), but
unfortunately the actual server response code is not available to our
code. It can be observed in the console output from the background page,
but programmatically, we only receive the WebSocket CloseEvent codes
listed here:
https://developer.mozilla.org/en-US/docs/Web/API/CloseEvent#Status_codes
The websocket error message is displayed by a normally-hidden but ever
present socket status element. Clicking this element will immediately
refresh the background page, which will try again to open the websocket
connection.
When first intalling, users will no longer be presented with the option
to register as a standalone client.
For developer convenience, the standalone form can still be found at
chrome-extension://.../register.html
Closes#159
Only re-render a message if the body changed. Re-render only the
delivery receipt checkmark if the delivered property changes.
Fix a bug where attachments flash in and out of existance when a
delivery receipt arrives.
Define a Whisper.View base class that automatically parses and renders
templates and attributes defined by the subclass. This saves us a good
number of lines of code as well as some marginal memory overhead, since
we are no longer saving per-instance copies of template strings.
Although I find the previous implementation more elegant, it results in
a deeper nesting of Promises than necessary, which can make debugging
more complicated. The canvas scaling and compression apis are actually
synchronous, so the callback structure isn't really recessary here.
Converting to a loop also makes this process easier to understand at
a glance.
Fixed some bugs along the way:
* accidentally scaling small images up to 1920px
* jpeg compressing gifs and other formats even if unnecessary
Previously we would not scale large resolution images with small file
sizes, but in fact, both resolution and file size constraints should be
enforced.
With these changes, message bubbles in the default-sized chat popup are
just wide enough to display the full complement of html5 media player
controls.
Converting attachment data to base64-encoded data uris takes O(n) and
there's no need! URL.createObjectURL returns a magic link that can be
set as the `src` attribute to `img`, `video`, and `audio` tags to load
blob data directly without copying.
https://developer.mozilla.org/en-US/docs/Web/API/URL/createObjectURL
Add contentType-specific limits, switch to lazy-init iff we encounter an
oversized file, and restyle as a toast, factoring out a generic
ToastView along the way.
Wait a little longer on initial scroll down. Previous timeout sometimes
triggered before all text is finished rendering.
Remove redundant resize calls.
Sometimes a conversation's messages would be reverse-ordered on first
load, correcting themselves after a refresh. This is an artifact of the
order we load messages from the database. To fix, load them in the
opposite order.
The alternative solution would be to reset the collection every time we
fetch new messages, but this would create an entirely new set of model
objects each time, which seems unnecessary.
Background page conversations were trying to trigger events on the inbox
list view which had been destroyed, resulting in a background page
console error of "can't read innerHeight of null".
Avoid this by removing listeners when the inbox window is closed.
When sending an constrct a copy of the PushMessageContent protobuf, add
a SyncMessageContext, and send it to ourselves. Do this for all kinds of
group messages, and individual text/media messages, but not closeSession
messages as the latter are device-specific.
Do not sync messages if we are the primary device, which should only be
supported in development. Normal web clients must be paired with a
android or ios master device, and even in dev, a primary/standalone web
client does not support linking additional devices.
The message view has three flavors so far, a normal text+attachments
message, a group update, and an end session message. This changeset
extracts the normal message rendering into its own subview, and adds
some convenience functions to the message model in order to simplify
some of that flavoring logic.
The first message sent to a new contact was throwing 'Unknown Group'.
This was because we didn't wait for the initial save to sync the `type`
attribute to indexedDB. Instead, don't trigger the conversation to open
until it has finished saving.
This is an artifact of a time when conversation elements would pop in
and out of the dom at a moment's notice, and thus needed to rebind their
event listeners regularly.
Previously the conversation window would query the background page
for a model id and then fetch the conversation. Instead, we can fetch
the conversation before opening the window, which simplifies the front
end scripts and avoids creating multiple copies of the same model.
Unless the background page fetches the latest details of a conversation
before updating it, it may clobber or nullify some attributes e.g., the
contact's name.
When a new message arrives, if its conversation is not already opened,
the background page opens it. If it is alrady open the window is
focused. Finally, the 'message' event is triggered, resulting in
1. the inbox refetches conversations
2. all conversations fetch new messages
TODO: only send this event to the target window
This collection is just an in-memory indexer used for typeaheads. For
display, the matching models are added to a separate collection. Thus,
the order of the elements in the typeahead collection does not matter.
It feels a little weird when you can't see the matching member. Would
consider putting this back in if we display the member list in the
contact list item view.
Previously, the ugly file input was hidden with opacity, and styled as a
square paperclip icon, but its drop and click zones were not constrained
to the visible square. They remained active across the whole 'Choose
File' button, which overlapped with the textarea. Instead, hide the file
input complete (display: none) and transmit click events from the
paperclip to the input programmatically.
Eventually, we'll need to address drag and drop events, but I want to do
that at the window level. Otherwise dropping a file outside the file
input drop zone causes the browser to navigate to the file://... url.
Render the entire conversation from a template, because some parts of it
must be rendered conditionally if it is a group vs private conversation.
Also apply some style fixes and restore lost functionality:
* Make conversation title bar fixed.
* Widens message bubbles.
* Unhide message list.
* Restore attachment rendering.
* Restore message sending and attachment file selection.
* Style attachments file input as a paperclip.
* Style send button like on Android and make it a submit input.
Don't auto open the last conversation. It doesn't make sense now that we
no longer have two column layout.
Don't trigger/listen for selected events. There's no need since the list
item opens a new popup now.
New private conversations have their type set in onMessageReceived. New
group conversations should be handled the same way as normal group
updates. It was pointed out we should never have to handle a group
message without a preceding group update, as those would be rejected by
textsecure.processDecrypted. An exception would be if you delete the
group from indexedDB but not localStorage, but that's not a mode we
should be supporting.
Also in this change I switched to instantiating a new conversation
object on every call to handlePushMessageContent. Originally, I thought
to use the local conversation list as a cache, but it's a bit simpler to
re-read from the database every time for now. Later on we should revisit
and optimize for fewer read/writes per incoming message.
Just display a sensible default in the frontend if it's unset.
For private conversations this should be the phone number, for
groups, the list of numbers.
This was intended to sync the group state of a recently re-installed
client, but is prone to overkill when we have a lot of old stale groups
around. Also this implementation incurs some rate limit errors from the
server.
Uses app-level timestamps for outgoing messages.
Adds timestamp property to the outgoing jsonData.
Triggers a runtime event to notify frontend on delivery receipts.
Renders delivered messages with a 'delivered' class.
This change removes the timestamp field from messages and conversations
in favor of multiple semantically named timestamp fields: sent_at,
received_at on messages; active_at on conversations. This requires/lets
us rethink and improve our indexing scheme thusly:
The inbox index on conversations will order entries by the
conversation.active_at property, which should only appear on
conversations destined for the inbox.
The receipt index will use the message.sent_at property, for effecient
lookup of outgoing messages by timestamp, for use in processing delivery
receipts.
The group index on conversation.members is multi-entry, meaning that
looking up any phone number in this index will efficiently yield all
groups the number belongs to.
The conversation index lets us scan messages in a single conversation,
in the order they were received (or the reverse order). It is a compound
index on [conversationId, received_at].
This ended up turning into a rewrite/refactor of the background page.
For best results, view this diff with `-w` to ignore whitespace. In
order to support retrying message decryption, possibly at a much later
time than the message is received, we now implement the following:
Each message is saved before it is decrypted. This generates a unique
message_id which is later used to update the database entry with the
message contents, or with any errors generated during processing.
When an IncomingIdentityKeyError occurs, we catch it and save it on the
model, then update the front end as usual. When the user clicks to
accept the new key, the error is replayed, which causes the message to
be decrypted and then passed to the background page for normal
processing.
ReplayableErrors make it easy for the frontend to handle identity key
errors by wrapping the necessary steps into one convenient little
replay() callback function.
The frontend remains agnostic to what those steps are. It just calls
replay() once the user has acknowledged the key change.
The protocol layer is responsible for registering the callbacks needed
by the IncomingIdentityKeyError and OutgoingIdentityKeyError.
superfeedr has done a nice job with this backbone -> indexedDB adapter,
but their query interface is somewhat limited. This commit adds an
alternate interface that lets us specify the index and cursor bounds we
want. This interface requires deeper knowledge of indexedDB indices, but
is more powerful overall.
This was used to conditionally render messages in the group style, but
it's actually unnecessary. We can render the same markup in both cases
and change the appearance with css.
This commit provides the javascript complement to
[WebSocket-Resources](https://github.com/WhisperSystems/WebSocket-Resources),
allowing us to use a bi-directional request-response framework over
websockets.
See websocket-resources.js and websocket-resources_test.js
for usage details.
Along the way I also factored the websocket keepalive and reconnect
logic into its own file/wrapper object.
Move base64 encoding of attachments to an AttachmentView. This makes
image rendering an asynchronous task so we fire an update event to
indicate to the parent MessageListView that its content has changed
height and it is time to scroll down.
Register the runtime callback at the top level view rather than having
each conversation view register independently.
Also refactors Layout into InboxView.
After a message is saved asynchronsly, fire an event and pass the
message attributes to frontend listeners via the chrome-runtime API.
This behavior is similar to the 'storage' event fired by localStorage.
Getting up and running with IndexedDB was pretty easy, thanks to
backbone. The tricky part was making reads and writes asynchronous.
In that process I did some refactoring on Whisper.Threads, which
has been renamed Conversations for consistency with the view names.
This change also adds the unlimitedStorage permission.
Eliminates the global Whisper.Messages object and consolidates shared
send/receive logic in Whisper.Threads.
To the latter end, note that the decrypted array buffer on an attachment
pointer is now named data instead of decrypted, in order to match the
format of outgoing attachments presented by
FileReader.readAsArrayBuffers and let us use the same handler to base64
encode them.
This dependency may be a little heavy for our current use case, but we can
roll with it for now and find something slimmer if it turns out yagni.
Closes#77Closes#40
Runtime reload is overkill and causes a jarring ux. Instead, send and
receive messages across the runtime. Also, if we need to jump between
the main ui and options pages, simply navigate within the current tab
rather than spawning a new one.
We only depend on cryptojs for this webcrypto polyfill, so let Grunt
concatenate them into one file.
The reference in the getString helper isn't needed since we use the
built in string converters on CryptoJS's word arrays.
Rename methods on the curve25519 interface to be a bit more high level.
Cleanup emscripten wrapper class, wrap long lines and such. Also add a
grunt task alias for building the emscripten compiled curve
implementation.
Firstly, don't initialize textsecure.nativclient unless the browser
supports it. The mimetype-check trick is hewn from nacl-common.js.
Secondly, nativeclient crypto functions will all automatically wait for
the module to load before sending messages, so we needn't register any
onload callbacks outside nativeclient.js. (Previously, if you wanted to
do crypto with native client, you would have to register a call back and
wait for the module to load.) Now that the native client crypto is
encapsulated behind a nice interface, it can handle all that
onload-callback jazz internally: if the module isn't loaded when you
call a nativeclient function, return a promise that waits for the load
callback, and eventually resolves with the result of the requested
command. This removes the need for textsecure.registerOnLoadCallback.
Finally, although native client has its quirks, it's significantly
faster than the alternative (emscripten compiled js), so this commit
also lets the crypto backend use native client opportunistically, if
it's available, falling back to js if not, which should make us
compatible with older versions of chrome and chromium.
Man you shoulda been there. Code was compiling all over the damn place.
It was wild.
Seriously though. Ignore that intermediate compiled file. What happens
in build stays in build.
Build with `grunt compile && grunt concat:curve25519` after installing
emscripten.
Enable by either (a) not loading nativeclient.js or (b) setting
`textsecure.NATIVE_CLIENT = false` before loading nativeclient.js.
The nativeclient.js module overrides
window.textsecure.registerOnLoadFunction with its own version. Otherwise
helpers will define a trivial placeholder for same.
The flag textsecure.NATIVE_CLIENT can be set anywhere ahead of
nativeclient.js, but is only acted on in nativeclient.js,
and crypto.js.
NB: this diff is best viewed with --ignore-whitespace
Distills crypto.js down to the hard cryptoey bones. It pulls from
webcrypto for aes and hmac, and from native client for curve25519 stuff
or potentially another object implementing the handful of needed
curve25519 functions.
Everything else formerly known as crypto, including session storage and
management, axolotl, etc.. is now protocol.js. The separation is not
quite perfect, but it's a big step.
nativeclient.js now enables talking to the native client module through
a high level interface as well as registering callbacks that will be
executed once the module is loaded. And it has tests!
Finally, this commit removes all references to the "testing_only"
object, preferring to run tests on textsecure.crypto instead.
To components. Because tab-completion works better when there aren't two
things starting with bower, and shorter names are nicer to deal with in
general.
Latest protobuf.js requires that we pass in the sign value when making
longs from strings, ex: dcodeIO.Long.fromString(id, true);
However, it does the string->long conversion automatically if its given
a string for a fixed64 field, so we can pass our string ids right in!
ftw
Moved all test code into /test. Renamed test.js to crypto_test.js.
(Let's try to keep test files topical.) Merged test_views.html and
test.html into a single test/index.html.
Todo: use Grunt to generate test/index.html from index.html and files
found in /test. Also, write more tests.
We now correctly and opportunistically use the webcrypto API if
available, polyfilling if it's not detected. This change also includes a
layer of abstraction over the webcrypto interface so we no longer have
to deal with key-imports or algorithm names all over the place. Since we
no longer support AES-CTR, code outside this file can simply call
`textsecure.subtle.<encrypt|decrypt|sign>(key, data [, iv])`.
TypedArray.prototype.set doesn't handle ArrayBuffers correctly (it
writes all zeros). Instead, wrap each ArrayBuffer in a typed array
for concatenation.
processData (default: true)
Type: Boolean
By default, data passed in to the data option as an object (technically,
anything other than a string) will be processed and transformed into a
query string, fitting to the default content-type
"application/x-www-form-urlencoded". If you want to send a DOMDocument,
or other non-processed data, set this option to false.
https://api.jquery.com/jQuery.ajax/
Parse attachment ids out of the attachment pointer url and return them
as strings because the copy parsed by JSON suffers a loss of precision.
Convert them to and from the format expected by the protobuf using
facilities from decodeIO.Long.
Sadly, we are not quite compliant with the WC3 webcrypto spec
due to our insistance on passing around key data in plain old
ArrayBuffers.
Also converted whitespace.
DRY up protobuf declarations and move to a slightly briefer naming
convention.
Also dropped some ArrayBuffer -> string conversions as
ProtoBuf.js handles ArrayBuffers just fine, and in fact, more
efficiently than strings.
Finally, dropped the btoa() wrappers, because that incurs an extra
string -> string conversion before the protobuf's internal string ->
array buffer conversion. In lieu of btoa, we can simply pass in the
optional string encoding argument to the protobuf's decode method,
which in these cases should be 'binary'.
Related: #17
All the group messages were being sent to the last recipient in the
list, due to the persistence of `var number` in later loops and async
calls. An easy mistake to make, when you use for instead of each.
An exception is thrown when protobuf tries to encode a number as the
group id, which is declared to have type 'bytes'.
Fix by make it an ArrayBuffer instead, and increase the length to 16,
which is what the Android client uses:
c632b32ff8/src/org/thoughtcrime/securesms/database/GroupDatabase.java (L222)
Previously, if calling createNewGroup with an undefined groupId,
no groupId was generated.
This occurred because no entry for "group" + undefined exists in
localStorage, which caused this code to think undefined was a
valid group id.
Fixed by adding `|| groupId == undefined` to the while clause.
Also decoupled the groupId collision check for clarity.
When codes are sent they are formatted as xxx-xxx. Previously when I
would paste these from GVoice they failed validation thanks to the dash
and whatever whitespace I happened to grab.
lilia
Patrick Connolly
Matt Corallo
Florian Groh
Matt Corallo
Riley Shaw
Emily Chao
Dave Sescleifer
Daniel Reichert
Sumit Bindal
Arlo Breault
Arnaud Benard