evernotearchive / zing Goto Github PK

The new delimiter headers display language names inconsistently. See the following screenshot as a reference:

When we perform TM queries to Elasticsearch, we need to provide the Content-Type header as part of the request. Note that this is enforced since ES version 6.

Creating projects from the UI triggers a synchronous file scan, which produces request timeouts and possibly other undesirable side-effects. That is a bug.

Since FS synchronization is meant to be run via sync/update_stores and this would be run continuously, we can safely remove file scanning upon project creation because any subsequent sync cycle after project creation will scan the FS and pick up the new files,

When stats are requetsed via /xhr/stats/, they are returned in an old format (dictionary) whereas we now expect an array.

We need to change TreeItem.get_stats and CachedTreeItem.get_stats to match the new code.

Additionally, I'd suggest to look why we have code duplication in TreeItem.get_stats and CachedTreeItem.get_stats

How to reproduce:

1. Shutdown RQ workers to prevent stats from updating
2. Change something in a unit (e.g. toggle the 'Needs work' flag)
3. Go back to the browser view

Since the data will be stale, the page will start requesting stats data via XHR, and this data will come in a wrong format, causing errors in the console (because React components get data in an unexpected format).

I've installed Zing for the first time following this guide https://evernote.github.io/zing/docs/initial-setup, and all projects (both built-in and added by me) seem to be empty:

When running update_stores the following errors appeared multiple times:

2018-12-13 15:16:48,240 ERROR Failed to run update_stores over /zh_CN/terminology/
Traceback (most recent call last):
  File "/Users/slukin/zing-env/lib/python2.7/site-packages/pootle/apps/pootle_app/management/commands/__init__.py", line 78, in do_translation_project
    process_stores = self.handle_translation_project(tp, **options)
  File "/Users/slukin/zing-env/lib/python2.7/site-packages/pootle/apps/pootle_app/management/commands/update_stores.py", line 49, in handle_translation_project
    force=options['force'], overwrite=options['overwrite'])
  File "/Users/slukin/zing-env/lib/python2.7/site-packages/pootle/apps/pootle_translationproject/models.py", line 320, in update_from_disk
    store.updater.update_from_disk(overwrite=overwrite)
  File "/Users/slukin/zing-env/lib/python2.7/site-packages/pootle/apps/pootle_store/updater.py", line 354, in update_from_disk
    store_revision=store_revision)
  File "/Users/slukin/zing-env/lib/python2.7/site-packages/pootle/apps/pootle_store/updater.py", line 279, in update
    user, submission_type,
  File "/Users/slukin/zing-env/lib/python2.7/site-packages/pootle/apps/pootle_store/updater.py", line 329, in update_from_diff
    updated, suggested, unsynced_uids = self.update_units(update)
  File "/Users/slukin/zing-env/lib/python2.7/site-packages/pootle/apps/pootle_store/updater.py", line 401, in update_units
    UnitUpdater(unit, update).update_unit()
  File "/Users/slukin/zing-env/lib/python2.7/site-packages/pootle/apps/pootle_store/updater.py", line 220, in update_unit
    self.save_unit()
  File "/Users/slukin/zing-env/lib/python2.7/site-packages/pootle/apps/pootle_store/updater.py", line 191, in save_unit
    self.unit.save(revision=self.update.update_revision)
  File "/Users/slukin/zing-env/lib/python2.7/site-packages/pootle/apps/pootle_store/models.py", line 442, in save
    self.store.update_dirty_cache()
  File "/Users/slukin/zing-env/lib/python2.7/site-packages/pootle/core/mixins/treeitem.py", line 469, in update_dirty_cache
    self.register_all_dirty()
  File "/Users/slukin/zing-env/lib/python2.7/site-packages/pootle/core/mixins/treeitem.py", line 428, in register_all_dirty
    r_con.zincrby(KEY_DIRTY_TREEITEMS, p)
TypeError: zincrby() takes exactly 4 arguments (3 given)

What should I try now? Running on MacOS 10.14.1, may this be related?

I was checking the AutoIcon component and saw that initial state is blindly initialized based on props:

  this.state = assign({
    saturation: 100,
    lightness: 30,
    abbreviationLength: 2,
    mode: 'solid',
    size: 20,
  }, props);

The component's state is assigned whatever props are passed to it. Note that using props in state is an anti-pattern.

Besides that, it looks like the component could get away with no state or almost no state at all, as these can perfectly be props. Generally speaking, if a component contains a lot of state, it's usually a sign of a code smell, and at the same time state should be kept at the absolute minimum, as the Thinking in React doc explains:

Simply ask three questions about each piece of data:

1. Is it passed in from a parent via props? If so, it probably isn't state.
2. Does it remain unchanged over time? If so, it probably isn't state.
3. Can you compute it based on any other state or props in your component? If so, it isn't state.

While checking #62, I realized that the cached information for the Last Updated column contains more information than we actually use.

With the new browsing table, we are only interested in the timestamp for the creation_time. The lastupdated key in the cache contains the following information though:

{
  "lastupdated": {
    "creation_time": <int:timestamp>,
    "display_datetime": <string:datetime>,
    "iso_datetime": <string:datetime>,
    "source": <string:unit-source>,
    "unit_url": <string:unit-permalink>,
  }
}

Unless there's a use-case I am missing, we should reduce this to a single value and hence make better use of the server memory. This would also simplify the initial queries for refreshing the data.

When you go to e.g. https://translate.evernote.com/projects/website4_evernote/dev-content/webroot/ and open the drop-down, you will see that there are no files under /dev-content/webroot/:

However, if you navigate to any language, e.g. https://translate.evernote.com/de/website4_evernote/dev-content/webroot/ you will see there are a bunch of .po files in this folder:

Steps to reproduce:

1. Create 4 priority tasks in some language. Reload the page to make sure the list of tasks is collapsed. You will see "1 more task" label (which is expected).
2. Open a separate browser tab and add one more priority task in the same language (in any translation project or folder)
3. Go back to the first tab and click "1 more task" label.

Expected: the list expands to show 5 unique tasks, the "1 more task" label diappears.

Observed: the list will expand and show 5 tasks, but:

• one of the task lines will appear in the list twice (incorrect)
• the just added task won't show up (incorrect)
• the label will now say "-1 more task" (incorrect)

The similar problem happens when a task is deleted in a side tab:

1. Create 4 priority tasks in some language. Reload the page to make sure the list of tasks is collapsed. You will see "1 more task" label (which is expected).
2. Open a separate browser tab and delete one of the priority tasks in the same language
3. Go back to the first tab and click "1 more task" label.

Expected: the list is updated to show only three relevant tasks; "1 more task" label disappears.

Observed: the list is expanded to show 4 tasks; "1 more task" label is still displayed.

We can try addressing this by re-fetching the task data on window focus.

In #37 we had to remove view tests that checked the context being rendered because they were basically duplicating application code and their usefulness is rather questionable.

Instead of this, we will rather use snapshot testing: take a snapshot of the rendered/returned context and store it in a file, serialized. This human-readable output will be checked by developers for validity and it will be committed to version control. Test runs will read from the snapshot file and compare it to the output provided by the view being tested.

Changes in view contexts/payloads mean the snapshot files need to be reviewed and updated too.

Steps to reproduce:

1. Set user account's full name to something like </script>Name<script>alert('Hi')</script>.
2. Open any page displaying Avatar components for this user, e.g. its profile page.
3. Hover over the avatar.

Observed behavior:
The script specified as a name is executed.

Expected behavior:
No script is run.

I believe this might be related to the tooltip (tipsy), but we will need to check.

Currently there are two forms to modify users:

1. UserForm for admin pages
2. EditUserForm for users to modify their profiles

There's some logic duplicated across the board, for instance to validate the linkedin field; that logic should be shared instead.

It looks like from time to time duplicate submissions are slipping in the DB.

The logs show for example the following:

[10/Apr/2017 02:50:43]  <username>  SC      18.0    TA      #<uid>        NS=18   S=0     (total: 336226.39639)
[10/Apr/2017 02:50:43]  <username>  A       <locale>      <uid> /<locale>/<project>/<resource>   <Translation>
[10/Apr/2017 02:50:43]  <username>  SC      18.0    TA      #<uid>        NS=18   S=0     (total: 336226.39639)
[10/Apr/2017 02:50:43]  <username>  A       <locale>      <uid> /<locale>/<project>/<resource>   <Translation>

So the same translation addition from the same user is being reported for unit uid. This is a duplicated submission that shouldn't happen.

In a production DB one can retrieve a list of units with duplicated submissions as follows (it might take a while to complete the query):

SELECT id, unit_id, creation_time, submitter_id, field, COUNT(*)
FROM pootle_app_submission
GROUP BY unit_id, creation_time, submitter_id, field
HAVING COUNT(*) > 1

Note this also lists rows which look like duplicates but are legit: e.g. quickly muting multiple quality checks for the same unit. Most of the time, duplicates refer to multiple consecutive unit submissions from the same user though (changes to state or target, or multiple suggestions).

We should ensure illegitimate duplicates cannot be created by:

1. setting a DB constraint on columns (creation_time, submitter_id, unit_id, field might not be enough).
2. avoiding any unit submissions from the UI when there is an ongoing submission.

We want to allow easy installation via Docker. This would also mean Docker becomes the sole officially-supported way to install Zing.

When navigating to a specific file (example link) we are no longer expanding advanced stats for the file, so the page looks a bit empty. This is a regression introduced in the new browser code.

Until we find a better way to utilize this view, let's return an automatic stats expansion.

Consider the following unit:

https://translate.evernote.com/kab/windows_v6_evernote/translate/entray/MsgBoxes.xml.po#unit=6265658

In the database, we have the following string:
Could not add tray icon, error: <Error />.

But in the editor UI it displayed like this:
Could not add tray icon, error: <Error ></Error>.

Because of that, translators use the same expanded <Error ></Error> form in the translation and get checks triggered. I believe this is a regression, as we use used to translate these strings in the past just fine.

Bugs like translate/pootle#4983 show that the current unit retrieval approach can't reliably work by design. We want to experiment with reverting the offset-related changes and at the same time use this as an opportunity to rethink the API between the client and the server and make this a foundation to the future translation UI improvements, e.g. continuous (non-paginated) scrolling. We also want to be considerate of the memory used by the browser, because this was the primary concern of the Pootle team with the old code.

Moving relevant bits from this comment here:

Let's keep the snapshot of uids on the client and not redo any search. It would work as follows:

1. Whenever the user goes into translation UI, server returns them a vector of uids capped at e.g. 1000 units (this is more than enough for one pass, be it review or translation). This adds about 10KB to the ungzipped page source, which should be ok.
2. Client renders the UI, and as user navigates through the units, it sends requests to the server specifying exact uids for the units it needs information about. The returned data is needed for view rows only, so it needs to contain only source and target text. If the unit is no longer there, it is returned as a blank entry in the results, which allows us to clean up these uids in the list.
3. When/if the user reaches the 1000th unit in a single pass, and if there were more than this number of units in the filter initially, the UI can render a button at the bottom of the table, by clicking on which the user can redo the search (reapply the filter) and start with unit 1. The effect of this button would be the same as going back to the browser table and then clicking the pill link again.

It seems that with the browser view refactoring we no longer display disabled project's contents to the admin. The data is being provided on the page, but the React component doesn't render the rows.

Currently we limit the number of units displayed in the export view to N (N=500), but it can be a bit short when files contain many short units. Translators have requested to uplift/remove this limit, however, should there be the need to do so, what we want to allow is pagination to load more units on demand.

Some recently-added components started using the FooBarBaz style for CSS class names, whereas we have always been using foo-bar-baz. Let's stick to the latter for consistency and switch the new ones to the traditional style.

Sometimes it may be beneficial to symlink to translation project directories instead of copying files back and forth. Right now symlinks confuse the update_stores mechanism, which seems to treat them as regular files.

For example, if one creates a directory symlink called foo, trying to register a new project with foo code will cause an error.

Similarly, if one registers the foo project first and then replaces the created foo directory with a symlink, the next time update_stores is run, the project will become disabled as if its project directory is gone.

1. Go to https://translate.evernote.com/da/website2_evernote/
2. Click on the Total header

Observed:
Rows with 0 values are intersected between rows with numbers higher than 0.

Expected:
0 values always come first in ascending order, last in descending order.

This works fine when there are no zeros involved, e.g. https://translate.evernote.com/da/

Export view's intended use-case is proofreading, which should be performed by registered users. Therefore we want to limit the access to the export view to registered users only.

When a unit contains submissions related to changes in quality checks (mute/unmute), these are properly reflected in the timeline.

However, when a new translation is provided (including an empty string), quality checks for that unit are deleted, and with that, due to the ON DELETE CASCADE behavior, the Submissions for the unit are deleted as well. After this, the timeline no longer reflects the past changes to muting/unmuting quality checks.

Changing the ON DELETE behavior to SET NULL wouldn't help because we need to have access to the quality check name, which is in the now-removed QualityCheck. We can alternatively denormalize the check name into Submission, but we should be aware that that requires increasing the size of the submissions table for a limited use case, plus schema and data migrations.

We are currently using the manifest feature of webassets/django-assets, but Django already provides a built-in solution for this: ManifestStaticFilesStorage. Let's switch to using that when possible, so that we depend less and less on webassets.

See the screenshot:

This happens in Firefox, Chrome and Safari (didn't check IE) and it seems to be caused by z-index. I'm leaving this for you @iafan as you are familiar with the latest changes that happened in the navbar.

Top scoring data has a dynamic nature and some caching aside, it is calculated approximately at run-time. This causes problems with snapshots, as the list of top scorers changes as time passes too.

We can either blacklist this key from our context data or somehow monkeypatch the date/time for the entire test suite (at the expense of missing on catching some bugs). There might be other options which I'm happy to hear about.

For now I'll go ahead and blacklist top_scorers from the snapshots because otherwise we'll be left with broken builds.

Since the move to the react-based browsing table, any user can click on stats numbers in /projects/ pages. This always leads to a 403 page for regular users, which is fine, however the links shouldn't be exposed to them.

We recently hit a bug in update/sync where the code was failing to run due to an ambiguous datetime. This happened because a) the server clock switched away from DST, and b) pytz is now available.

What this outlined is that for unknown reasons update/sync is converting timestamps into timezone-aware datetime objects. Since unix timestamps work perfectly fine in this scenario, so the conversion is unnecessary.

Steps to reproduce:

1. Go to a unit with a suggestion
2. Click on it, and modify its content in the form
3. Click on Accept

Observed:
The translation is placed in the textarea, but it is not saved. In fact, it blanks any existing translations.

Expected:
The translation is placed in the textarea and saved to the DB.

It seems that the textarea is populated but the UI is not sync'ed, hence the value read when performing the submission with edits is blank and an empty string is sent in the POST payload (target_f_0=).

When changing the title or body field of a static page, its modified_on timestamp should automatically be updated, however this is not the case and the timestamp will be left intact upon save.

The current way legal page agreements are checked is highly inefficient, because a query is run on every non-xhr request for logged in users. This is implemented as a middleware.

This can be improved to check for such condition after users have been authenticated (both in normal and social accounts) but before signing them in. Modifying legal pages could invalidate existing sessions to force accepting any new/updated conditions, should that be needed

Going through a ToS acceptance flow should redirect users back to the page that signing in would redirect them to.

As a start, and to avoid confusion, the LEGALPAGE_NOCHECK_PREFIXES setting from staticpages can be moved to a constant.

Webpack 2 is around the corner! 🎉

This is a big shift from webpack 1 which might need some tweaking in the config but with the help of the migration guide it shouldn't be big trouble.

Among the benefits of using webpack 2, we will have means to auto-remove dead exports (tree shaking) and ship less code to users, which also means less parsing to be done on their side.

In some of the newly-added components, a lot of strings are missing i18n markers (t(), nt() and so on), so they are effectively not ready for localization.

Among others, this includes HelpDialogContent and AboutDialogContent.

Steps to reproduce:

1. Open the editor in a context where there are no failing checks, but trigger the checks UI with .../translate/#filter=checks. At this point no failing checks are displayed.
2. Change the filter to something else, e.g. All
3. Perform a translation which triggers any type of check failure.
4. Select Checks from the filtering options.

Observed:
The failing check is not displayed in the drop-down.

Expected:
The failing check is displayed in the drop-down.

Note this was also the case before (as already commented in #80 (comment)) but now it becomes a bit more prominent as selecting Checks allows to browse through all checks, and the UI expands too.

The problem is that elements are removed from the DOM (motivated because hiding option elements doesn't work reliably across browsers), so after that event they won't be available again until a full page refresh. Ideally we should be able to declaratively specify the check groups and checks we want to display, and don't worry about the DOM at all.

There is a synchronization bug where:

1. Unit U is translated (or somehow changed) in store X, and it is still NOT synced to disk
2. The file for store X ceases to exist on disk
3. update_stores is run, making all units in store X obsolete
4. The file for store X is brought back to the previous location on disk
5. update_stores is run, and units from the file are resurrected, except those which contain unsynced changes such as unit U.

In some places .toLocaleString() has been introduced.

This method accepts a locale code as its first parameter (even an array of locales for matching). We must set this parameter to the UI locale in effect and probably provide a thin wrapper formatting function so that we get the same experience everywhere. Otherwise, the OS locale will always be used, and has high chances to mismatch Zing's UI locale.

When one follows some link to a unit which is no longer available (example), the UI doesn't update properly: the wheel keeps spinning, and there's no 'Unit not found' message.

The permissions admin pages contain dropdowns for users which load the entire (!!) user set in one go. The list should be populated asynchronously as admins type in a desired username.

When rebuilding parts of the UI in React, we want to take advantage of the locally-scoped CSS rules tied to React components, and this use this as an opportunity to clean up monolithic CSS we currently have.

Let's use PostCSS for this and integrate it with Webpack so that we can produce final CSS bundles upon saving individual component CSS files.

Hi there,

I have tried to install Zing for the last few hours and cannot seem to seamlessly install Zing and have it working. I will describe what I have done so that you could reproduce it.
I religiously followed all the steps described in here. Once I run
bash (env) $ zing runserver --insecure
the server starts correctly. However, once I access the URL in my browser, I get the following error:

(env) $ zing runserver --insecure
Performing system checks...

System check identified some issues:

WARNINGS:
?: (pootle.W004) Email server is not available.
	HINT: Review your email settings and make sure your email server is working.
?: (pootle.W006) The sqlite database backend is unsupported
	HINT: Set your default database engine to postgresql or mysql
?: (pootle.W010) DEFAULT_FROM_EMAIL is using the following default setting 'info@YOUR_DOMAIN.com'.
	HINT: DEFAULT_FROM_EMAIL is used in all outgoing Pootle email.
Don't forget to review your mail server settings.
?: (pootle.W011) POOTLE_CONTACT_EMAIL is using the following default setting 'info@YOUR_DOMAIN.com'.
	HINT: POOTLE_CONTACT_EMAIL is the address that will receive messages sent by the contact form.

System check identified 4 issues (0 silenced).
December 21, 2017 - 12:46:59
Django version 1.11.8, using settings 'pootle.settings'
Starting development server at http://127.0.0.1:8000/
Quit the server with CONTROL-C.
Traceback (most recent call last):

 [...]

  File "/home/robin/dev/newest_zing/env/local/lib/python2.7/site-packages/webassets/bundle.py", line 334, in get_version
    'determined dynamically, because: %s') % (self, reason))
BundleError: Cannot find version of <Bundle output=css/common.min.%(version)s.css, filters=[<webassets.filter.cssmin.CSSMin object at 0x7f5a5d71e810>], contents=('css/style.css', 'css/actions.css', 'css/breadcrumbs.css', 'css/buttons.css', 'css/contact.css', 'css/error.css', 'css/auth.css', 'css/magnific-popup.css', 'css/navbar.css', 'css/odometer.css', 'css/popup.css', 'css/react-select.css', 'css/tipsy.css', 'css/zing-icons.css', 'css/select2.css', 'css/select2-pootle.css', 'css/scores.css', 'css/user.css', 'css/welcome.css')>. There is no manifest which knows the version, and it cannot be determined dynamically, because: output target has a placeholder
[2017-12-21T12:47:04]	"GET / HTTP/1.1" 500 59
2017-12-21 12:47:04,103 ERROR "GET / HTTP/1.1" 500 59

After some googling, I found that the same error happened for some Pootle installations, and it seems related to the fact that the assets are not built. http://docs.translatehouse.org/projects/pootle/en/latest/developers/customization.html#customization

So I executed:

(env) $ zing webpack

which returned the following error:

CommandError: webpack executable not found.
Make sure to install it by running `cd /home/robin/dev/newest_zing/env/local/lib/python2.7/site-packages/pootle/static/js && npm install`

so I executed the command:

cd /home/robin/dev/newest_zing/env/local/lib/python2.7/site-packages/pootle/static/js && npm install

and finally,

(env) $ cd -
(env) $ zing webpack
(env) $ zing collectstatic --noinput --clear -i node_modules
(env) $ zing assets build

After all this is done, I managed to run the server by doing
(env) $ zing runserver --insecure
which started the server correctly and once I access localhost:8000, I get the following view with the following errors in the console

and if I tried to click on Sign in, I would get this other error:

Uncaught ReferenceError: gettext is not defined
    at t.getScreen (http://localhost:8000/assets/js/common/app.min.ed8b2ec8.js:1:227825)
    at t.render (http://localhost:8000/assets/js/common/app.min.ed8b2ec8.js:1:228654)
    at l._renderValidatedComponentWithoutOwnerOrContext (http://localhost:8000/assets/js/vendor.min.efc4ede4.js:1:195026)
    at l._renderValidatedComponent (http://localhost:8000/assets/js/vendor.min.efc4ede4.js:1:195114)
    at l.mountComponent (http://localhost:8000/assets/js/vendor.min.efc4ede4.js:1:191388)
    at Object.mountComponent (http://localhost:8000/assets/js/vendor.min.efc4ede4.js:1:131002)
    at Object.obj.(anonymous function) [as mountComponent] (chrome-extension://fmkadmapgofadopljbjfkapdkoienihi/build/backend.js:8598:20)
    at l.mountComponent (http://localhost:8000/assets/js/vendor.min.efc4ede4.js:1:191484)
    at Object.mountComponent (http://localhost:8000/assets/js/vendor.min.efc4ede4.js:1:131002)
    at Object.obj.(anonymous function) [as mountComponent] (chrome-extension://fmkadmapgofadopljbjfkapdkoienihi/build/backend.js:8598:20)
getScreen @ app.min.ed8b2ec8.js:1
render @ app.min.ed8b2ec8.js:1
_renderValidatedComponentWithoutOwnerOrContext @ vendor.min.efc4ede4.js:1
_renderValidatedComponent @ vendor.min.efc4ede4.js:1
mountComponent @ vendor.min.efc4ede4.js:1
mountComponent @ vendor.min.efc4ede4.js:1
obj.(anonymous function) @ backend.js:8598
mountComponent @ vendor.min.efc4ede4.js:1
mountComponent @ vendor.min.efc4ede4.js:1
obj.(anonymous function) @ backend.js:8598
mountComponent @ vendor.min.efc4ede4.js:1
mountComponent @ vendor.min.efc4ede4.js:1
obj.(anonymous function) @ backend.js:8598
mountComponent @ vendor.min.efc4ede4.js:1
mountComponent @ vendor.min.efc4ede4.js:1
obj.(anonymous function) @ backend.js:8598
u @ vendor.min.efc4ede4.js:1
perform @ vendor.min.efc4ede4.js:1
l @ vendor.min.efc4ede4.js:1
perform @ vendor.min.efc4ede4.js:1
batchedUpdates @ vendor.min.efc4ede4.js:1
batchedUpdates @ vendor.min.efc4ede4.js:1
_renderNewRootComponent @ vendor.min.efc4ede4.js:1
obj.(anonymous function) @ backend.js:8588
_renderSubtreeIntoContainer @ vendor.min.efc4ede4.js:1
render @ vendor.min.efc4ede4.js:1
open @ app.min.ed8b2ec8.js:1
on.t @ app.min.ed8b2ec8.js:1
dispatch @ vendor.min.efc4ede4.js:1
v.handle @ vendor.min.efc4ede4.js:1

I am a bit at a loss as to what to do next. I use Ubuntu 16.04 in case that's useful.

Looking forward to hearing from you!

Best wishes

If one translate all the units that were filtered out and finally hits Submit button on the last unit, we "collapse" this unit (display it as a view row instead of the edit row). However, if one now clicks on that last unit to expand the editor again, nothing will happen. Clicking on previous units, if there are any, works as expected (the editor will show up).

1. Go to https://translate.evernote.com/projects/
2. Click on the Last updated column to sort by the last update

Observed result:
Sorting fails when reaching undefined values (see screenshot).

Expected result:
Rows are sorted based on the last update.

When the source unit contains one or more ellipsis symbols (…) and the translation contains a series of three or more dots (...), we should trigger an 'ellipsis' check.

Reason: this kind of check can be found in other linters, and units that fail it can trigger build errors.

Some export view pages are limited the same way as translate pages. This was due to performance considerations and to control server load.

At the same time, it's useful to share links with non-admin users that lists strings in export view representing a query. The export view already limits the result set quite considerably though, and performance of such pages is good enough to grant access to any type of user, so we want to experiment with it to enable it for everyone in this context.

We are currently requiring emails to be unique via allauth but it must also be enforced at the DB level. Note how the meta users (system, nobody, default) also need to have an email address assigned to them.

To reproduce:

1. build website with make docs from project's root directory
2. open website/build/zing/index.html.

All links, including those to pages, images and css files are abolute and start with /zing/.... This prevents them from loading and the website doesn't look the way it looks when deployed.

Fixing this would really help with working on documentation.

Today we had an issue with some translation being 'stuck' on the translation server. While translation was provided 4 days ago, it wasn't downsynced to .po files, and running sync_stores command (without --force) wouldn't export it either.

So I looked at the logs and here is the relevant log entries around the time the submission was made:

[...]
[17/Dec/2018 01:15:49]  system  X       ./manage.py sync_stores --skip-missing --project=webclipper_evernote
[17/Dec/2018 01:15:51]  system  X       ./manage.py update_stores --project=webclipper_evernote
[17/Dec/2018 01:15:53]  system  X       ./manage.py sync_stores --skip-missing --project=website3_evernote
[17/Dec/2018 01:15:54]  nijinik SC      12.8571428571   TX      #7077486        NS=18   S=0     (total: 167295.04494)
[17/Dec/2018 01:15:54]  nijinik C       ja      7077486 /ja/website4_evernote/cms-content/feeds/notes.json.po   目標へ向かって前進していくなかで、その週にう>
[17/Dec/2018 01:15:58]  system  X       ./manage.py update_stores --project=website3_evernote
[17/Dec/2018 01:16:00]  system  X       ./manage.py sync_stores --skip-missing --project=website4_evernote
[17/Dec/2018 01:16:02]  [sync] File saved; updated 1 units in /ja/website4_evernote/cms-content/features/search-handwriting.json.po [revision: 2047766]
[17/Dec/2018 01:16:02]  [sync] File saved; updated 3 units in /ja/website4_evernote/cms-content/feeds/notes.json.po [revision: 2047774]
[17/Dec/2018 01:16:02]  [sync] File saved; updated 3 units in /ja/website4_evernote/cms-content/templates/index.json.po [revision: 2047769]
[17/Dec/2018 01:16:26]  nijinik SC      11.0    TA      #7077483        NS=11   S=0     (total: 167307.902083)
[17/Dec/2018 01:16:26]  nijinik A       ja      7077483 /ja/website4_evernote/cms-content/feeds/notes.json.po   課題を特定し、どのように目標を達成するかを具>
[17/Dec/2018 01:16:29]  system  X       ./manage.py update_stores --project=website4_evernote
[17/Dec/2018 01:16:31]  [update] updated 5 units in /ja/website4_evernote/cms-content/feeds/notes.json.po [revision: 2047776]
[17/Dec/2018 01:16:35]  system  X       ./manage.py sync_stores --skip-missing --project=windows_evernote
[17/Dec/2018 01:16:37]  system  X       ./manage.py update_stores --project=windows_evernote
[17/Dec/2018 01:16:39]  system  X       ./manage.py sync_stores --skip-missing --project=windows_v6_evernote
[...]

Key events and my interpretation:

1. [01:16:00] - sync_stores started on website4_evernote
2. [01:16:02] - /ja/.../notes.json.po file was saved to disk [revision: 2047774]
3. [01:16:26] - translation for unit 7077483 was provided in /ja/.../notes.json.po file (I assume it got revision 2047775)
4. [01:16:29] - update_stores started on website4_evernote
5. [01:16:31] - 5 units were read from /ja/.../notes.json.po file [new revision: 2047776]

So after that translation with revision 2047775 was never synced back to .po (we were thinking that .po file is already at revision 2047776).

One way to address this is, at update_stores time, to check if the current revision of the file is the same as the last saved one for that file. If revisions differ, skip bumping up the revision. This will guarantee that next time sync_stores is executed, it will save all the new units back.

I understand that we introduced revision bumping at update_stores time as an optimization, to avoid re-exporting units that came from .po files. But in rare cases like the one described above, we can sacrifice this optimization.

In #138 we made sure stats are made available to the test functions that require it by leveraging the refresh_stats fixture. The implementation might seem more like a workaround though.

We need to come up with a more robust solution that ensures predictability while not sacrificing speed.

Bringing here some ideas outlined in the original PR thread:

Improve TreeItem's testability

• Make TreeItem operations no-op by default in the test environment.
  This removes the chance for accidental poisoning. We can also think of alternative ways to make this class easier to test.
• Individual tests that require R/W access to stats can explicitly request so (via a custom marker, for instance), hence circumventing the default behavior.

Imitate rollback via code

• Redis has DUMP and RESTORE commands that might come handy. If we know that a specific test is going to alter certain keys, we can wrap it with the decorator that will backup keys before they are modified with the application code, and then restore the modified keys once the test is finished.
• This will allow us to actually test how the stats are affected during tests.

At least with Chinese characters this doesn't look good. It's probably also an issue with CJK overall, and some other scripts can be affected too. Check out the following screenshot:

Unfortunately I'm not sure what's the best approach here as I ignore the language rules and potential connotations.