View source for Community Tech/Ebook Export Improvement/tr

{{Community Tech/Hatnote|done}}

<languages/>
<div lang="en" dir="ltr" class="mw-content-ltr">
The Ebook Export Improvement project aims to improve the experience of exporting books from Wikisource. Under the current system, users struggle with a variety of issues, including reliability, formatting, styles, and user experience. These issues add complexity and frustration to the Wikisource process, and they discourage some users from deeply engaging with Wikisource. This project was the [[Special:MyLanguage/Community Wishlist Survey 2020/Wikisource/Improve export of electronic books|#1 request]] from the [[Special:MyLanguage/Community Wishlist Survey 2020|2020 Community Wishlist Survey]].
</div>

<div lang="en" dir="ltr" class="mw-content-ltr">
Overall, Wikisource ebook exports have tremendous potential, but they must be improved in order to serve a wider audience. In the course of this project, we’ll aim to investigate and identify the key issues, collaborate with various Wikisource communities, and implement solutions that further sustain and improve ebook exports. '''We look forward to [[Talk:Community Tech/Ebook Export Improvement Project#Project Overview: Request for Feedback (May 2020)|community feedback on the Talk page]].'''
</div>

<div lang="en" dir="ltr" class="mw-content-ltr">
== Why Export Ebooks ==
</div>

<div lang="en" dir="ltr" class="mw-content-ltr">
Generally speaking, ebook exports are a core part of the Wikisource experience, and users export ebooks for a variety of reasons. First, they may export ebooks to avoid issues with internet accessibility. With offline books, users can easily read materials on a variety of devices, no matter if their internet connection is slow, intermittent, or unreliable.
</div>

<div lang="en" dir="ltr" class="mw-content-ltr">
Second, users may want to read the book on a device that is optimized for ebooks. For example, devices such as the Kindle or Kobo allow users customize the interface, add notes, look up words, and have a long battery life. Consequently, many users choose to export an ebook into a compatible file format, which can then be transferred to an eReader device.
</div>

<div lang="en" dir="ltr" class="mw-content-ltr">
Third, users may want to share an ebook with someone in an easily accessible format. For this reason, they may choose to export the ebook in a preferred format, which they can share with the recipient. This process is generally more flexible than sharing a web link to Wikisource.
</div>

<div lang="en" dir="ltr" class="mw-content-ltr">
Fourth, the user may work in a professional setting, such as an educational institution, archive, or museum, and they want to store an offline copy for general reference or educational purposes. The export of an ebook allows them to integrate the content into their own files and workflow, rather than having to consistently access Wikisource.
</div>

<div lang="en" dir="ltr" class="mw-content-ltr">
== How Ebook Exports Work ==
</div>

<div lang="en" dir="ltr" class="mw-content-ltr">
In Wikisource, there are four primary methods to download an ebook: directly via WSExport, via the left side panel, via links on the main page (such as a featured book), or via the links above texts. There is also a fifth method (create a book), which is technically available, but very uncommonly used. We will discuss all of the methods below. However, it is important to understand that not all wikis have the same download options. Some have only one download option, while others have many options.
</div> 

<div lang="en" dir="ltr" class="mw-content-ltr">
=== #1: WSExport ===
</div>

<div lang="en" dir="ltr" class="mw-content-ltr">
First, [https://ws-export.wmcloud.org/ WSExport] is the primary tool for exporting ebooks on Wikisource. This tool, originally developed by user [[oldwikisource:User:Tpt|Tpt]] for French Wikisource, permits downloads in a variety of formats, such as EPUB and PDF. To access WS Export, one can navigate to the left side-panel and click on “Choose format,” under the “Download/print” section. Alternatively, users can directly visit [https://ws-export.wmcloud.org/ ws-export.wmcloud.org]
</div>

<div lang="en" dir="ltr" class="mw-content-ltr">
When using WSExport, the user must specify certain things to generate an ebook export. These include the language code, the title of the page, fonts included (if any), and whether images should be included. The user must manually type in the language code and page name, but they can select the file format and fonts via drop-down. Once the user clicks “Export,” the tool will provide a downloadable version of the page in the user’s specified format. Further documentation on the tool can be found on [[oldwikisource:Wikisource:WSexport|Wikisource:WSexport]].
</div>

<div lang="en" dir="ltr" class="mw-content-ltr">
It should be noted that the “Include fonts” section has different use cases, depending on the language selection. For Latin-scripted languages, the user is generally selecting a font, such as “Free Serif.” However, for Indic languages, the user must often specify an actual script, which is required to properly export the ebook.
</div>
[[File:Wsexport_basic_screenshot.png|alt=|none|thumb|600x600px|<span lang="en" dir="ltr" class="mw-content-ltr">Example of WSExport page, when exporting an EPUB file in English with no fonts</span>]]

<div lang="en" dir="ltr" class="mw-content-ltr">
=== #2: Export via the left side panel ===
</div>

<div lang="en" dir="ltr" class="mw-content-ltr">
Second, users can access ebook exports via the left side panel (“Download/print”). With this method, the user can see all file formats available for download. For example, in the screenshot below, the user has the option to download a PDF, EPUB, or MOBI file of ''The Jungle''.
</div> 

<div lang="en" dir="ltr" class="mw-content-ltr">
When using the side panel, the default option available is typically only single-part PDFs, which are downloaded via [https://www.npmjs.com/package/electron-pdf ElectronPDF.] If you want to download a multi-part book in the side panel, you need to enable WSExport in Preferences > Gadgets > Interface > “Add a print/export link to download pages as EPUB files using the WSExport tool.” This will add the EPUB and MOBI option to the navigation menu, so that you can directly download multi-part books. In the example below, you can see that the user can download the entire contents, since they have enabled multi-part book downloads.
</div>
[[File:Thejungle_cropped_outline.png|none|thumb|600x600px|<span lang="en" dir="ltr" class="mw-content-ltr">Example of multi-part book, which can be downloaded as an EPUB file</span>]]

<div lang="en" dir="ltr" class="mw-content-ltr">
=== #3: Export via links on the main page ===
</div>

<div lang="en" dir="ltr" class="mw-content-ltr">
Third, users can download books via links on the main page. For example, in the screenshot below, you will see that “April’s Featured Article” has a section called “Grab a download!” One can choose from four main file formats, which are specified if the user hovers over each icon. This user can click an icon, which automatically triggers a download that uses the WSExport tool.
</div>
[[File:Downloadviafeaturedtext.png|none|thumb|600x600px|<span lang="en" dir="ltr" class="mw-content-ltr">Example of direct download icon links on main page (English Wikisource)</span>]]

<div lang="en" dir="ltr" class="mw-content-ltr">
=== #4: Export via links at the top of text ===
</div>

<div lang="en" dir="ltr" class="mw-content-ltr">
Fourth, users can sometimes access download links at the top of text. For example, in the screenshot  below, download options are presented for a text in Bengali Wikisource. Like the side panel, the user sees the file formats that are available for download, and the downloads are conducted with the WSExport tool.
</div>
[[File:Featured_Text_Bengali_Wikisource.png|none|thumb|600x600px|<span lang="en" dir="ltr" class="mw-content-ltr">Example of direct download links at the top of the page (Bengali Wikisource)</span>]]

<div lang="en" dir="ltr" class="mw-content-ltr">
=== #5: “Create a book” ===
</div>

<div lang="en" dir="ltr" class="mw-content-ltr">
Fifth, you can create a book as an ODT and ZIM file. To do this, go to the “Create a book” link in the side panel, which will redirect you to Special:Book, also known as “Manage your book.” On this page, you can manually specify each page that you want in your book, which you will need to do repeatedly until you have specified all the pages. It should be noted that this process was designed for Wikipedia users who want to create source texts. It is not convenient for Wikisource users, so it is rarely used for Wikisource-related purposes.
</div>

<div lang="en" dir="ltr" class="mw-content-ltr">
== The Primary Issues with Current Ebook Exports ==
</div>

<div lang="en" dir="ltr" class="mw-content-ltr">
There are many issues with the current ebook export process, which we will divide into three categories: reliability, formatting and styles, and user experience.
</div>

<div lang="en" dir="ltr" class="mw-content-ltr">
=== Reliability ===
</div>

<div lang="en" dir="ltr" class="mw-content-ltr">
The WSExport tool is not consistently reliable. Users report frequent downtime and timeout issues, which prevent them from exporting books. Some of these issues have been documented in Phabricator tickets, such as [[phab:T250614|T250614]] and [[phab:T219330#5060262|T219330#5060262]]. In 2019, this issue became the [[Special:MyLanguage/Community Wishlist Survey 2019/Wikisource/Improve export of electronic books|#4 wish]] in the [[Special:MyLanguage/Community Wishlist Survey 2019|2019 Community Wishlist Survey]]. As a result, the Community Tech team launched the [[Special:MyLanguage/Community Tech/E-Book Export Reliability|Ebook Export Reliability project]], which aimed to improve the export experience. By the [[Special:MyLanguage/Community Tech/E-Book Export Reliability#June 26, 2019|completion of the project]], WSExport had 99.42% average monthly update (recorded on June 20, 2019), and downtime went from 941 minutes total (between May 1-15, 2019) to 179 minutes (between June 1-15, 2019). Further data on Wikisource reliability after the team’s changes can be found in [[phab:T226136|T226136]].
</div>

<div lang="en" dir="ltr" class="mw-content-ltr">
Despite these improvements, issues persisted for the Wikisource community. Users continued to experience issues, such as those detailed [[s:fr:Utilisateur:Viticulum/Anomalies_epub/english|Problems detected in epub generated with Wsexport]], compiled by [[s:fr:Utilisateur:Viticulum|Viticulum]] on French Wikisource. For example, on September 30, 2019, WSExport was inoperable for about 12 hours. In total, 13 outages were reported, with many of them lasting for hours. Furthermore, recent tests conducted by the Community Tech team have identified intermittent problems when trying to download MOBI files. The timeout message currently only appears when on the WSExport page (not via download links), as well. In total, this situation can be very frustrating for Wikisource users, and they often don’t know how to respond to such issues.
</div>

<div lang="en" dir="ltr" class="mw-content-ltr">
=== Formatting & Styles ===
</div>

<div lang="en" dir="ltr" class="mw-content-ltr">
Ebook exports often have formatting and style issues. These issues vary, but they may include: missing or altered text, duplicated text, poor pagination, missing table titles, incorrect capitalization, incorrect border styles, incorrect content alignment, incorrect table alignment, and incorrect table styles. In some cases, the words themselves are altered. These errors can be confusing and concerning to users. They also go against the Wikisource policy of mirroring the source text. Below, we have provided some examples of the issues we’re seeing. This isn’t an exhaustive list, but it can give some idea of common errors.
</div>

<div lang="en" dir="ltr" class="mw-content-ltr">
====''Example #1: Page split between 2 pages''====
</div>

<div lang="en" dir="ltr" class="mw-content-ltr">
In the screenshot below, you will see that the content is divided into two pages. However, in the [[wikisource:Trees_and_Other_Poems|original version]] in English Wikisource, it was displayed on one page.
</div>
[[File:Wikisource_cutoff1.png|alt=|left|page=cut off in English wikisource (pg 1)|thumb|423x423px|<span lang="en" dir="ltr" class="mw-content-ltr">Content split into 2 pages (1st page)</span>]]
[[File:Wikisource_cutoff2.png|alt=|none|page=cut off in English wikisource (pg 1)|thumb|425x425px|<span lang="en" dir="ltr" class="mw-content-ltr">Content split into 2 pages (2nd page)</span>]]
{{-}}

<div lang="en" dir="ltr" class="mw-content-ltr">
====''Example #2: Fonts not rendered''====
</div>

<div lang="en" dir="ltr" class="mw-content-ltr">
In the screenshots below, you will see that the files are not properly exported from Tamil Wikisource (bottom left) or [[s:kn:%E0%B2%B5%E0%B2%BF%E0%B2%AE%E0%B3%8B%E0%B2%9A%E0%B2%A8%E0%B3%86|Kannada Wikisource]] (bottom right). Rather, the text displays as rectangles. This is due to the fact that Kannada is not included in the “Include fonts” section, which creates various issues, such as the one below. Meanwhile, Tamil is included in “Include Fonts,” but there are still issues.
</div>
[[File:Tamili_Wikisource_First_page_Screenshot.png|alt=|left|thumb|428x428px|<span lang="en" dir="ltr" class="mw-content-ltr">First page of book exported from Tamil Wikisource, which fails to render the characters, even after including Tamil fonts.</span>]]
[[File:Kannada_Wikisource_ebook_export_issue_(screenshot).png|alt=|none|thumb|427x427px|<span lang="en" dir="ltr" class="mw-content-ltr">Kannada Wikisource. The font is not rendered because the Kannada font is not available.</span>]]
{{-}}

<div lang="en" dir="ltr" class="mw-content-ltr">
====''Example #3: Consonant conjuncts incorrectly rendered''====
</div>

<div lang="en" dir="ltr" class="mw-content-ltr">
In the example below, you will see that the text is incorrectly displayed. In the [[s:bn:%E0%A6%9A%E0%A6%BE%E0%A6%81%E0%A6%A6%E0%A7%87%E0%A6%B0_%E0%A6%AA%E0%A6%BE%E0%A6%B9%E0%A6%BE%E0%A6%A1%E0%A6%BC|original version]] in Bengali Wikisource, the user will see “প্রথম.” However, in the ebook export, the word is changed to “পরথম.” This particular error shows [[:en:Conjunct_consonant|conjunct consonants]], which is when two consonants are usually clustered together in a word. In these examples, the consonants are separated due to issues with font rendering. This issue occurs for users in many Indic languages.
</div>
[[File:Consonant conjunct issue in wikisource.png|none|thumb|400x400px|<span lang="en" dir="ltr" class="mw-content-ltr">Consonant conjuncts incorrectly rendered in Wikisource</span>]]
{{-}}

<div lang="en" dir="ltr" class="mw-content-ltr">
====''Example #4: Incorrect text wrap''====
</div>

<div lang="en" dir="ltr" class="mw-content-ltr">
In the example below, you will see that the text is wrapped around the image. However, in the [[s:bn:%E0%A6%9A%E0%A6%BE%E0%A6%81%E0%A6%A6%E0%A7%87%E0%A6%B0_%E0%A6%AA%E0%A6%BE%E0%A6%B9%E0%A6%BE%E0%A6%A1%E0%A6%BC/%E0%A6%AA%E0%A7%8D%E0%A6%B0%E0%A6%A5%E0%A6%AE_%E0%A6%AA%E0%A6%B0%E0%A6%BF%E0%A6%9A%E0%A7%8D%E0%A6%9B%E0%A7%87%E0%A6%A6|original version]] on Bengali Wikisource, the text is displayed below the image with no text-wrap.
</div>
[[File:Incorrectextwrap.png|none|thumb|424x424px|<span lang="en" dir="ltr" class="mw-content-ltr">Example of incorrect text wrap (text should be below image)</span>]]
{{-}}

<div lang="en" dir="ltr" class="mw-content-ltr">
====''Example 5: Content alignment altered''====
</div>

<div lang="en" dir="ltr" class="mw-content-ltr">
In this example below, you will see that content is aligned to the left in the PDF. However, in the [[s:hy:%D5%8D%D5%A1%D5%BC%D5%A1%D5%B6|original text]] from Armenian Wikisource, the content is centered on the page.
</div>
[[File:Armenian Wikisource content alignment issue.png|none|thumb|425x425px|<span lang="en" dir="ltr" class="mw-content-ltr">Content aligned to the left, rather than centered</span>]]

<div lang="en" dir="ltr" class="mw-content-ltr">
=== Accessibility & User Experience ===
</div>

<div lang="en" dir="ltr" class="mw-content-ltr">
The ebook export process is not very inviting to newcomers. There are many quirks and exceptions that one must learn. The WSExport tool is not easily discoverable, and it doesn't provide an intuitive user experience. For example, it doesn’t include all scripts in the “Include fonts” section (such as Bengali). Even if a language script is included in “Include fonts,” the export may still experience language errors. Meanwhile, in the sidebar, it’s confusing to determine how to download multi-part ebooks for new users, among other issues. If we want Wikisource to expand, we need the experience to be intuitive and accessible newcomers. For this reason, the UX considerations involved in the ebook export process may be investigated for improvement as well.
</div>

<div lang="en" dir="ltr" class="mw-content-ltr">
== Open Questions ==
</div>
* <span lang="en" dir="ltr" class="mw-content-ltr">Have we covered the main reasons why people export ebooks?</span>
* <span lang="en" dir="ltr" class="mw-content-ltr">Have we covered the main methods to export ebooks?</span>
* <span lang="en" dir="ltr" class="mw-content-ltr">Have we covered the main problems experienced when exporting ebooks?</span>
* <span lang="en" dir="ltr" class="mw-content-ltr">Which formatting and style issues are the most common and frustrating, in your opinion?</span>
* <span lang="en" dir="ltr" class="mw-content-ltr">Which user experience issues are the most common and frustrating, in your opinion?</span>
* <span lang="en" dir="ltr" class="mw-content-ltr">Which problems, overall, do you find the most critical to fix, and why?</span>
* <span lang="en" dir="ltr" class="mw-content-ltr">Anything else you would like to add?</span>

<div lang="en" dir="ltr" class="mw-content-ltr">
'''''We look forward to reading [[Talk:Community Tech/Ebook Export Improvement Project#Project Overview: Request for Feedback (May 2020)|your feedback on the Talk page]]! Thank you!'''''
</div>

<span id="Status_Updates"></span>
== Durum Güncellemeleri ==

<div lang="en" dir="ltr" class="mw-content-ltr">
=== March 31, 2021: Final update ===
</div>

<div lang="en" dir="ltr" class="mw-content-ltr">
Hello, everyone! We are now thrilled to announce that the Ebook Export Improvement project is complete. This has been a tremendously rewarding experience, and we have really enjoyed collaborating with the diverse and passionate Wikisource community. Below, we will share a summary of the work we did and the impact it has had. Thank you again for all of your collaboration!
</div>

<div lang="en" dir="ltr" class="mw-content-ltr">
==== Reliability ====
</div>

<div lang="en" dir="ltr" class="mw-content-ltr">
We knew that, in order for our work to be impactful, Wikisource Export needed to be a more reliable and efficient tool. For this reason, we focused on improving the overall health of the code that supports the tool. We also tackled multiple issues that were contributing to slow or error-laden downloads. The technical details of this work can be found in previous updates. Now, we’re excited to share the final results:
</div>

* <span lang="en" dir="ltr" class="mw-content-ltr">'''The [https://stats.uptimerobot.com/BN16RUOP5/782558466 Uptime Robot Dashboard] shows that WS Export is nearly always up (as of 31 March 2021):'''</span>
** <span lang="en" dir="ltr" class="mw-content-ltr">Last 24 hours: 99.139%</span>
** <span lang="en" dir="ltr" class="mw-content-ltr">Last 7 days: 99.768%</span>
** <span lang="en" dir="ltr" class="mw-content-ltr">Last 30 days: 99.834%</span>
** <span lang="en" dir="ltr" class="mw-content-ltr">Last 90 days: 99.896%</span>
* <span lang="en" dir="ltr" class="mw-content-ltr">'''The median export time for books is 3 seconds''', as found from one month of data, recorded between 17 February to 17 of March ([[phabricator:T274013|source]]). We don’t know the median export time before our changes, since we only recently began logging this data.</span>
* <span lang="en" dir="ltr" class="mw-content-ltr">'''There are many more exports being successfully generated:'''</span>
** <span lang="en" dir="ltr" class="mw-content-ltr">Within a 30 day period, our data recorded 1,168,667 successful exports after our changes ([[phabricator:T274013|source]]) vs. 71,613 successful exports before our changes ([[phabricator:T256018|source]]). ''This shows 16 times as many successful exports after the changes.''</span>
** <span lang="en" dir="ltr" class="mw-content-ltr">Within a 60 day period, our data recorded 1,513,486 successful exports after our changes ([[phabricator:T274013|source]]) vs. 160,433 successful exports before our changes ([[phabricator:T256018|source]]). ''This shows 9.4 times as many successful exports after the changes.''</span>
** <span lang="en" dir="ltr" class="mw-content-ltr">Within a 90 day period, our data recorded 1,628,523 successful exports after our changes ([[phabricator:T274013|source]]) vs. 249,924 successful exports before our changes ([[phabricator:T256018|source]]). ''This shows 6.5 times as many successful exports after the changes.''</span>
** <span lang="en" dir="ltr" class="mw-content-ltr">Note: It makes sense that the greatest growth happened within 30 days, since we have released the most changes within the last few months. However, it should be noted that some of the ebook exports (both before and after the changes) may be from bots, and we cannot easily determine which downloads are from bots.</span>
* <span lang="en" dir="ltr" class="mw-content-ltr">'''The error rate of ebook exports is significantly lower:'''</span>
** <span lang="en" dir="ltr" class="mw-content-ltr">We have decreased the likelihood of encountering an ebook error by approximately 12 times (within a 30 day period). Here is the breakdown of the data:</span>
*** <span lang="en" dir="ltr" class="mw-content-ltr">Before our changes, we recorded a total of about 46,000 errors per 71,613 successful exports within a 30 day period. This was calculated because we had found a total of 22,982 errors in a 14-day period, which we can roughly double to be about 46,000 errors in a 30 day period ([[phabricator:T256018|source]]). This means that, for every 1.5 downloads, there was one exception thrown.</span>
*** <span lang="en" dir="ltr" class="mw-content-ltr">After our changes, we recorded a total of 63,921 exception errors per 1,168,667 successful exports within a 30 day period ([[phabricator:T274013|source]]). This means that, for every 18 successful ebook exports, there was 1 exception thrown. This is a 12 times improvement over the previous ratio. This is based on the data recorded in a 30 day period after our changes.</span>
*** <span lang="en" dir="ltr" class="mw-content-ltr">Note: There isn’t a 1:1 relationship between exports and exceptions. In other words, one unsuccessful export can generate multiple exception errors.</span>
* <span lang="en" dir="ltr" class="mw-content-ltr">'''We expect that the error rate will drop even lower very soon:'''</span>
** <span lang="en" dir="ltr" class="mw-content-ltr">In our new data, the most common error type is DriverException, which accounts for 46% of all exception errors. This is a new error type, which isn’t even present in our old data. This is because this error type was brought on by our [[wikitech:Wiki_Replicas|replicas work]].</span>
** <span lang="en" dir="ltr" class="mw-content-ltr">There is already a fix for DriverException errors that will be deployed soon. Once this fix is released, the total error rate should be even lower.</span>
* <span lang="en" dir="ltr" class="mw-content-ltr">'''We have added a new option for problematic exports: download without credits.'''</span>
** <span lang="en" dir="ltr" class="mw-content-ltr">We determined that one common cause of slow or problematic book downloads was the credits displayed in the “About” section. The credits can take a lot of time to generate and may cause ebook export issues. For this reason, you can now choose to “Exclude editor credits” in the Wikisource Export page.</span>

<span id="Language_Support"></span>
==== Dil Desteği ====

<div lang="en" dir="ltr" class="mw-content-ltr">
The original wish focused on WS Export reliability. However, as we consulted with Wikisource communities, we learned about serious language support issues. We made it a project priority to fix these issues. We believe in the power of Wikisource as a global, equitable tool of free knowledge, so it’s absolutely crucial that it’s inclusive of all languages. Here are the results:
</div>

* <span lang="en" dir="ltr" class="mw-content-ltr">'''We upgraded font support in ebook exports:''' We [[phabricator:T261479|fixed the issue]] that previously displayed boxes rather than characters in ebook exports, especially for non-Latin script languages. Now, everyone can download books in their chosen language!</span>
* <span lang="en" dir="ltr" class="mw-content-ltr">'''The Wikisource Export page has been internationalized:''' The page used to be in English only. [[phabricator:T255224|We internationalized it]], so it would be available for translation, and it’s now been translated into many languages already.</span>
* <span lang="en" dir="ltr" class="mw-content-ltr">'''Communities can pick their default fonts for the download button:''' When we added the new download button, we heard from community members that they wanted the ability to choose the default font that was most suitable for them. We understood the importance of this, so [[phabricator:T274561|we made this possible]].</span>

<span id="User_Experience"></span>
==== Kullanıcı Deneyimi ====

<div lang="en" dir="ltr" class="mw-content-ltr">
As we wrote above, the original wish focused on reliability issues. However, as we consulted with community members and conducted usability tests, we learned that Wikisource Export needed user experience improvements as well. This way, it could be used (and enjoyed!) by a larger, more diverse group of people. For this reason, we issued a series of improvements. Here are the results:
</div>

* <span lang="en" dir="ltr" class="mw-content-ltr">'''There is now a user-friendly “Download” button:''' We simplified the user experience by [[phabricator:T271869|providing a user-friendly “Download” button]]. We drew from community feedback and usability testing to develop the feature. We also incorporated [[phabricator:T274999|updates to the tool]] after sharing the first iteration with community members, such as adding a link to “Other formats” on the WS Export page (thanks for the tip, [[User:Inductiveload|InductiveLoad]]!).</span>
* <span lang="en" dir="ltr" class="mw-content-ltr">'''We simplified the language code process:''' In the old WS Export page, users needed to manually enter in the language code, which was confusing for newcomers. With our [[phabricator:T269344|changes]], no language code knowledge is required! We provide all of the available languages in the dropdown.</span>
* <span lang="en" dir="ltr" class="mw-content-ltr">'''We improved support when errors occur:''' We have implemented greater support for users who encounter ebook export errors. Now, when users encounter errors, they will see [[phabricator:T277237|suggestions for next steps]] displayed on the Wikisource Export page. We also worked to ensure that the [[phabricator:T278189|error messaging is appropriate]] for the type of error that has occurred.</span>
* <span lang="en" dir="ltr" class="mw-content-ltr">'''The Wikisource Export page has been revamped:''' The older version of the Wikisource page was in need of a visual make-over. For this reason, our designer provided a series of recommendations to improve readability, usability, and mobile accessibility. The end result is a much cleaner design, and we hope you love it as much as we do!</span>

<div lang="en" dir="ltr" class="mw-content-ltr">
Overall, this project is now done. We thank you all for being such fantastic ambassadors and advocates for Wikisource, and we learned so much from all of you! If you have any questions or comments, we invite you to [[Talk:Community_Tech/Ebook_Export_Improvement#Final_project_update:_March_31,_2021|share them on the Talk page]]. We sincerely hope that we helped improve Wikisource Export, which is at the heart of the Wikisource experience. Up next, we’ll be working on another Wikisource project: [[Special:MyLanguage/Community_Tech/OCR_Improvements|OCR Improvements]] (which we encourage you to check out, if you haven’t already). Thank you everyone, and we look forward to further fruitful collaboration!
</div>

<div lang="en" dir="ltr" class="mw-content-ltr">
=== March 8, 2021: Updates on recent work ===
</div>

<div lang="en" dir="ltr" class="mw-content-ltr">
Hello, everyone! We are excited to share with you another update on our recent work on the project. Also, apologies for the delay since our last update! We have been very busy over the past few months with the holiday season, the [[Special:MyLanguage/Community_Wishlist_Survey_2021|2021 Community Wishlist Survey]], launching the [[Special:MyLanguage/Community_Tech/OCR_Improvements|Wikisource OCR Improvements project]], and additional team work. Please check out our update below and share your feedback on the project talk page. Thank you in advance!
</div>

<div lang="en" dir="ltr" class="mw-content-ltr">
==== Recently completed work ====
</div>

* <span lang="en" dir="ltr" class="mw-content-ltr">'''[[phabricator:T271869|Implement a new download button]]:''' Now when you go to a book on Wikisource, you will see a blue button, labeled “Download.” When you click on the button, a pop-up will appear that enables you to download books in PDF, EPUB, and MOBI formats. You will see information below the file format name, which lets you know which file format is appropriate for your device. We decided to do this work after collecting feedback from the user talk page and on usertesting.com, which confirmed that many people found the current download experience not very user-friendly. This new download button is meant to increase the accessibility of ebook exports on Wikisource.</span>
** <span lang="en" dir="ltr" class="mw-content-ltr">Note: We implemented a simplified version of the initial mockups. Some parts of the original proposal turned out to be too big and complex, so we opted for a trimmed down version, which you can now see on the wikis.</span>
* <span lang="en" dir="ltr" class="mw-content-ltr">'''[[phabricator:T255224|Internationalize the WS Export page]]:''' The Wikisource Export page used to be in English only, which was obviously a problem. With this change, the page can now be translated into many different languages. Translations have already begun. If you don’t see it translated into your wiki’s language, we invite you to help the translation effort on translatewiki!</span>
* <span lang="en" dir="ltr" class="mw-content-ltr">'''[[phabricator:T255790|Replace ElectronPDF with WSExport PDF support]]:''' Some wikis previously used ElectronPDF for PDF downloads in their sidebar links. ElectronPDF does not work very well, especially for Wikisource, which led to many people having frustrating download experiences. We replaced ElectronPDF links with WSExport links, so now all wikis can download PDFs via the sidebar with a more reliable service.</span>
* <span lang="en" dir="ltr" class="mw-content-ltr">'''[[Talk:Community_Tech/Ebook_Export_Improvement#Have_we_covered_the_main_methods_to_export_ebooks?|Migrate WSExport Gadget to to the Wikisource Extension]]:'''  In the process of consulting with Wikisource users on the [[Talk:Community_Tech/Ebook_Export_Improvement#Have_we_covered_the_main_methods_to_export_ebooks?|project talk page]], we learned that some users did not have the ability to click download links (such as "Download as MOBI") via the side panel and other places. This was the case on Czech Wikisource, for example. This was because these wikis did not have the WSExport gadget enabled on their wiki. To fix this issue, we migrated the gadget over to the Wikisource extension, so that all users can now have the same download links available.</span>
* <span lang="en" dir="ltr" class="mw-content-ltr">'''[[phabricator:T265660|Cache all API Requests]]:''' With this work, we have added another way to improve efficiency of ebook exports. Now, if someone downloads Book A and then someone else wants to download Book A shortly afterward, the generated ebook will already be cached. This means it doesn’t need to be newly regenerated for each request, and the download time will be much quicker the second time around.</span>
* <span lang="en" dir="ltr" class="mw-content-ltr">'''[[phabricator:T264788|Migrate API to Parsoid API]]:''' This work was recommended by [[User:Tpt|Tpt]], and we agreed with the recommendation, after conducting our own investigation. With this work, we now have better long-term support of formatted text on Wikisource exports. Also, this work was inevitable, since Parsoid API will eventually replace the current MediaWiki's native parser. Now that this work is done, we can identify bugs and issues early on, which we can direct to the Parser team. Please note that we are aware that some bugs came out of this work, but since the migration was scheduled to occur either way, we think it’s best to identify the bugs early on.</span>
* <span lang="en" dir="ltr" class="mw-content-ltr">'''[[phabricator:T270387|Reinstate OPDS Support]]:''' With this support reinstated for OPDS now reinstated, you can now access up-to-date catalogs of Wikisource books that are suitable for download on your device. It should be noted that OPDS export can be set up to work with any category on a wiki. It produces a daily updated file that can be used in software like Calibre, FBReader, and some e-readers to more quickly and easily browse and download available books. The existing [[d:Q102234502|categories]] are just the first ones; others can be made, even multiple per wiki (e.g. "Literature for export" or something).</span>
* <span lang="en" dir="ltr" class="mw-content-ltr">'''[[phabricator:T274575|Investigation on how to improve error messaging]]:''' We know that many people feel frustrated when there are ebook export errors, since there is typically minimal information or support. The purpose of this investigation was to determine what potential improvements we could implement from a technical perspective. Now that this work is complete, we are now looking into implementing improved support (see next section for details).</span>
* <span lang="en" dir="ltr" class="mw-content-ltr">'''Various bug fixes:''' In software development, whenever new work is done,  new bugs arise. As a team, we have analyzed and fixed many bugs, such as [[phab:T274957|RTL language support issues]], [[phabricator:T270372|internal links issues]], and other bugs. However, as mentioned above, some bugs may be handled by other teams in the future, such as Parsoid-related bugs (which may go to the Parser team).</span>

<div lang="en" dir="ltr" class="mw-content-ltr">
==== What's next ====
</div>

<div lang="en" dir="ltr" class="mw-content-ltr">
We are in the last stages of this project. Here’s the remaining work:
</div>

* '''<span lang="en" dir="ltr" class="mw-content-ltr">[[phab:T269344|Improve language selection on the Wikisource Export page]]:</span>'''
** <span lang="en" dir="ltr" class="mw-content-ltr">STATUS: In development. Engineers already have work they are reviewing.</span>
** <span lang="en" dir="ltr" class="mw-content-ltr">SUMMARY: Right now, users need to know the language code of the book (such as “en” for English or “fr” for French) in order to export it on the Wikisource Export page. This is not user-friendly. For this reason, we are implementing a selector, so you can pick the appropriate language from a dropdown</span>
* '''<span lang="en" dir="ltr" class="mw-content-ltr">[[phab:T274561|Allow communities to choose appropriate fonts for download]]:</span>''' 
** <span lang="en" dir="ltr" class="mw-content-ltr">STATUS: In development. Engineers already have work they are reviewing.</span>
** <span lang="en" dir="ltr" class="mw-content-ltr">SUMMARY: Wikisource users want a way to select which font is best for their language when downloading books. So, for example, if I am on Hindi Wikisource and I click the blue download button and then choose to download a book, the book should be in the font that the community decided is best for the Hindi language. We are now working to make this possible. This work is already in development.</span>
* '''<span lang="en" dir="ltr" class="mw-content-ltr">[[phab:T257886|Remove dependency on credit generation]]:</span>'''
** <span lang="en" dir="ltr" class="mw-content-ltr">STATUS: In development. Engineers already have work they are reviewing.</span>
** <span lang="en" dir="ltr" class="mw-content-ltr">SUMMARY: When this work is complete, wikis  will be able to download books without editor credits, if they want. We decided to do this work because credit generation is often a very intensive process. It can significantly slow down ebook exports. For this reason, if a user is experiencing difficulties downloading a book, they may want to try downloading the book without the credits. We wanted to implement the technical support to make this possible, either for users now or in the future.</span>
* '''<span lang="en" dir="ltr" class="mw-content-ltr">[[phab:T274959|Create option to disable credits]]:</span>''' 
* <span lang="en" dir="ltr" class="mw-content-ltr">STATUS: Ready for development and in engineer's backlog. Will be worked on very soon.</span>
** <span lang="en" dir="ltr" class="mw-content-ltr">SUMMARY: Once we have removed the dependency on credit generation, we can create the option to allow ebook exports without credits on the Wikisource Export page. This could be useful in cases when books are particularly difficult to download due to lag time and errors. We could also create a separate page with a list of the book credits, so that even users who choose to download books without the credits can view them within Wikisource Export.</span>
* '''<span lang="en" dir="ltr" class="mw-content-ltr">[[phab:T276488|Improve messaging about errors]]:</span>'''
** <span lang="en" dir="ltr" class="mw-content-ltr">STATUS: In planning stages.</span>
** <span lang="en" dir="ltr" class="mw-content-ltr">SUMMARY: Our investigation found that it would be too much work to implement specific messaging about errors within the pop-up. For this reason, we have decided to improve the error messaging on the Wikisource Export page. The basic idea is that, if you encounter an error via the download button, you will be redirected to the Wikisource Export page. There, you will be told that there was an error and be given proposed next steps. While this may not generate a successful ebook export for all use cases, it will provide more helpful messaging and support for users than the current default behavior.</span>

<div lang="en" dir="ltr" class="mw-content-ltr">
==== Open Questions ====
</div>

* <span lang="en" dir="ltr" class="mw-content-ltr">What do you think of the new "Download" pop-up? Do you find it easy to use?</span>
* <span lang="en" dir="ltr" class="mw-content-ltr">What do you think of our additional work to improve reliability? Are you still seeing improvements in the speed and performance of WS Export?</span>
* <span lang="en" dir="ltr" class="mw-content-ltr">What do you think of our remaining work to disable credits as an option?</span>
* <span lang="en" dir="ltr" class="mw-content-ltr">What do you think of our remaining work to improve messaging and support when there are download errors?</span>
* <span lang="en" dir="ltr" class="mw-content-ltr">Is there anything else you would like to add?</span>

'''''[[Talk:Community_Tech/Ebook_Export_Improvement#Project_updates:_Request_for_feedback_(March_2021)|<span lang="en" dir="ltr" class="mw-content-ltr">Please share your feedback on the project talk page!</span>]]'''''

<div lang="en" dir="ltr" class="mw-content-ltr">
=== November 18, 2020: Updates & Request for Feedback ===
</div>

<div lang="en" dir="ltr" class="mw-content-ltr">
Hello, Wikisourcers! We are pleased to share our November update. In this update, we’ll focus on the work we have done so far to improve the ebook export experience. We’ll also share our plans for what we hope to do next and how we plan to do it. We would like to thank everyone for the feedback so far, and [[Talk:Community_Tech/Ebook_Export_Improvement#Request_for_feedback_on_UX_mockups_&_general_updates_(November_2020)|we look forward to collecting feedback on this next stage of the project!]]
</div>

<div lang="en" dir="ltr" class="mw-content-ltr">
==== Language support improvements ====
</div>

<div lang="en" dir="ltr" class="mw-content-ltr">
You can now read Wikisource ebook exports that were previously unreadable. For example, issues such as [[#Example_#2:_Fonts_not_rendered|this]] and [[phab:T258128|this]] are now solved, due to our changes! Here’s how we did this: After investigating the issue, we realized that WS-Export only supported 4 main fonts. This meant that fonts required to properly render many scripts were not available.
</div>

<div lang="en" dir="ltr" class="mw-content-ltr">
For this reason, [[phab:T261479|we upgraded font support]], so that all fonts available in Debian would be available for WS-Export. The end result is that scripts in ebook exports are now largely supported, due to this change. One request: Now that we have made this change, please test out this issue for us. Do you think it is largely resolved? Do you still see boxes (instead of text) popping up anywhere? Please let us know, since we really want to fix this issue. Thanks in advance!
</div>

<div lang="en" dir="ltr" class="mw-content-ltr">
==== Reliability improvements ====
</div>

<div lang="en" dir="ltr" class="mw-content-ltr">
===== #1. Investigation findings =====
</div>

<div lang="en" dir="ltr" class="mw-content-ltr">
We have conducted four separate investigations to identify how we can improve ebook export reliability. These investigations were as follows: [[phab:T257902|Parsoid HTML for WS-Export]], [[phab:T222936|Cache generation for ebooks]], [[phab:T264122|Preventing automated book downloads]], and [[phab:T253283|Implementing a job queue system]]. In each case, an engineer was assigned to deeply explore the proposal in question, determine if and how we could make such changes, and what level of improvement we could expect from such changes.
</div>

<div lang="en" dir="ltr" class="mw-content-ltr">
Then, we reviewed the findings as a team to determine next steps. From these investigations, we decided that two proposals should be focused on first: Parsoid HTML and Caching ebooks. In both cases, we felt that the proposed changes would improve reliability and that they were within scope for the team. Work on both of these projects has been launched, which we’ll explain below.
</div> 

<div lang="en" dir="ltr" class="mw-content-ltr">
===== #2. Caching API requests =====
</div>

<div lang="en" dir="ltr" class="mw-content-ltr">
This [[phab:T265660|work]] came out of one of our investigations, as described above. We have made great progress, and it is almost complete. When the work has been deployed, we will cache all API requests when exporting Wikisource ebooks. This means that, if someone downloads Book A and then someone else wants to download Book A soon afterward, the ebook will be generated much faster.
</div>

<div lang="en" dir="ltr" class="mw-content-ltr">
This could be helpful in a variety of cases, such as when  many people download featured books of the month, when many people download books listed on wiki pages (such as those featured on the homepage [[s:bn:%E0%A6%89%E0%A6%87%E0%A6%95%E0%A6%BF%E0%A6%B8%E0%A6%82%E0%A6%95%E0%A6%B2%E0%A6%A8:%E0%A6%AA%E0%A7%8D%E0%A6%B0%E0%A6%A7%E0%A6%BE%E0%A6%A8_%E0%A6%AA%E0%A6%BE%E0%A6%A4%E0%A6%BE|on Bengali Wikisource]]), or when people sequentially download different formats of the same book. We believe that this work can help improve reliability, but we’ll need to conduct some analysis to determine the impact. We’ll share more on the release and impact in our next update.
</div>

<div lang="en" dir="ltr" class="mw-content-ltr">
===== #3. Integrate Symfony 5 with WS-Export =====
</div>

<div lang="en" dir="ltr" class="mw-content-ltr">
This [[phab:T263577|work]] is now complete. Due to this work, we are now able to implement improvements to WS-Export, such as the caching work, on a much faster timeline. Furthermore, Symfony 5 is the same framework we use for other tools, so ongoing maintenance will be easier and quicker in the future. We can also now make use of Symfony components that are battle-tested and require very little work to enable, such as ErrorHandler, Cache, Console, DependencyInjection, DotEnv, and more. Overall, we are modernizing the app and making it easier on ourselves and volunteers in the long run.
</div>

<div lang="en" dir="ltr" class="mw-content-ltr">
===== #4. Migrate API to Parsoid API =====
</div>

<div lang="en" dir="ltr" class="mw-content-ltr">
This [[phab:T264788|work]] is in progress and almost complete. Once complete, this work will help us support proper formatting, such as in footnotes and mathematical equations. It was first recommended to us by [[User:Tpt|Tpt]], and our investigations concluded that we should do the work. Here are the technical details: WS-Export was using the MediaWiki parser HTML output using ?action=render to generate its ePubs.
</div>

<div lang="en" dir="ltr" class="mw-content-ltr">
However, [[mw:Special:MyLanguage/Parsoid|Parsoid]] HTML became available, and it provides much richer data. Additionally, Parsoid API will eventually replace the current MediaWiki's native parser. We want Ws-Export to be up-to-date with the latest parser sooner rather than later. Once this work is complete, HTML output will hopefully be simplified, which will make it easier for us to support formatting of text.
</div>

<div lang="en" dir="ltr" class="mw-content-ltr">
==== User experience improvements: feedback requested ====
</div>

<div lang="en" dir="ltr" class="mw-content-ltr">
As we shared in our last status update, we repeatedly heard from people that the download experience is confusing, inconsistent, and hard to understand. We agree, and we want to improve the user experience. For this reason, we conducted an investigation to determine how we could improve the user experience, which included team discussions and usability tests. From this investigation, we came up with the mockups (as shown below) and ideas for how we can improve the experience. We encourage you to check them out and share your feedback on the talk page.
</div>

<div lang="en" dir="ltr" class="mw-content-ltr">
===== Proposed changes =====
</div>

<div lang="en" dir="ltr" class="mw-content-ltr">
'''General proposal:''' We propose to add or replace (depending on the wiki) the top download links on the page with a simple “Download” button. When the user clicks on the download button, a window will open up, which will list the download options for the book (such as, EPUB, MOBI, and PDF) with download icon links. The user will also see information about which file format is recommended for different device types (since we found in our user research that many users don’t know which file format to pick). Once they pick a file format, the download process will begin, which will be indicated by some sort of “in progress” indicator. When the download is completed, the user will see a status indicator to display that it is complete. If there is an error in downloading the book, the user will see an error status indicator.
</div> 
[[File:Wikisource Download Flow - Pick format.jpg|alt=|none|thumb|600x600px|]]

<div lang="en" dir="ltr" class="mw-content-ltr">
'''Potential enhancement for the future -- Auto-download:''' Once we have implemented the basic behavior, we may consider creating an auto-download function. Under this scenario, if the user has never downloaded a book before, they will manually pick the file format. After that, when the user clicks “Download,” the download will be automatically triggered in the file format that the user last selected. You can see an example of the user flow below.
</div>
[[File:Wikisource Download Flow - Auto-download.jpg|alt=|none|thumb|600x600px|]]

<div lang="en" dir="ltr" class="mw-content-ltr">
===== Usability tests =====
</div>

<div lang="en" dir="ltr" class="mw-content-ltr">
After we developed the mockups, we conducted some usability tests (with readers who did not know about Wikisource) to see how people responded to the mockups. We'll be updating the mock-ups based on our findings from the tests, as well as the feedback from all of you provided on the talk page.
</div>

<div lang="en" dir="ltr" class="mw-content-ltr">
==== Open questions ====
</div>

* <span lang="en" dir="ltr" class="mw-content-ltr">What do you think of our recent font support work? Does the issue seem to be largely resolved of boxes appearing rather than text?</span>
* <span lang="en" dir="ltr" class="mw-content-ltr">What do you think of our recent and upcoming reliability work? Do you have any thoughts, concerns, or suggestions to share?</span>
* <span lang="en" dir="ltr" class="mw-content-ltr">What do you think of our proposed improvement to the download user experience overall? Do you like the general idea and user flow?</span>
* <span lang="en" dir="ltr" class="mw-content-ltr">Do you usually download the same file format (e.g., PDF, MOBI, EPUB, etc) every time you download a book, or do you often pick a different format?</span>
* <span lang="en" dir="ltr" class="mw-content-ltr">Is there anything else you would like to share?</span>

'''''[[Talk:Community_Tech/Ebook_Export_Improvement#Request_for_feedback_on_UX_mockups_&_general_updates_(November_2020)|<span lang="en" dir="ltr" class="mw-content-ltr">Please share your feedback on the project talk page!</span>]]'''''

<div lang="en" dir="ltr" class="mw-content-ltr">
=== October 23, 2020: Engineering work has begun ===
</div>

<div lang="en" dir="ltr" class="mw-content-ltr">
The engineers have begun taking on work related to improved reliability of the WS-Export tool and improved font support. This work is based off the proposals that we shared on August 11th, as well as the feedback from that update. We are also wrapping up our first stage of research related to improving the user experience, which we'll be sharing soon.
</div>

<div lang="en" dir="ltr" class="mw-content-ltr">
=== August 11, 2020: Early findings ===
</div>

<div lang="en" dir="ltr" class="mw-content-ltr">
Hello, everyone! We are very excited to share our first update on the project. We want to thank everyone who has shared feedback on the project talk page, GitHub, or Phabricator so far. Your insights have been critical, and we deeply appreciate them. Once you have read our August update, we invite you to '''[[Talk:Community_Tech/Ebook_Export_Improvement#Early_findings:_Request_for_feedback_(August_2020)|share your feedback on the project talk page]]'''.
</div>

<div lang="en" dir="ltr" class="mw-content-ltr">
==== Lessons from the consultation (so far) ====
</div>

<div lang="en" dir="ltr" class="mw-content-ltr">
On the project talk page, we discussed with some people how to think about the project overall–in other words, what sort of principles should we be internalizing and following as we begin this work? Here’s what we have found so far:
</div>

# <span lang="en" dir="ltr" class="mw-content-ltr">'''We should be continually mindful of both contributors and visitors.''' These groups may have some overlapping needs, but there may also be some important differences for us to investigate, explore, and address.</span>
# <span lang="en" dir="ltr" class="mw-content-ltr">'''We should think about user experience improvements (rather than only technical improvements).''' The process to export ebooks is currently not intuitive for many users. If we can improve this experience, we can potentially retain and nurture a larger base of readers and editors.</span>
# <span lang="en" dir="ltr" class="mw-content-ltr">'''We should try to investigate, document, and share best practices for Wikisource.''' As we have consulted with people, we have learned that some perceived errors are actually not technically “bugs.” Instead, they are rooted in confusion over formatting and best practices. As such, they can potentially be fixed by the community. This is great news, but more people need visibility into these best practices. For this reason, we plan to investigate best practices, and we’ll share our findings with the community.</span>
# <span lang="en" dir="ltr" class="mw-content-ltr">'''We won’t be able to fix everything (unfortunately!), but we’ll document all the issues we encounter.''' For this project, we’ll be focusing on improving core accessibility (i.e., WSExport functionality & reliability) and core readability (i.e., people’s ability to read exported books in their language of choice). If there are issues that don’t fall into these two categories, we’ll still document the issues in Phabricator–but we may not be able to fix them. It is our hope that other people may be able to fix the issues that we can’t get to in the future.</span>

<div lang="en" dir="ltr" class="mw-content-ltr">
==== Work we have done so far ====
</div>

<div lang="en" dir="ltr" class="mw-content-ltr">
We have just launched the project, so we haven’t done much work yet. However, we have done some work, which we would like to share now:
</div>

# <span lang="en" dir="ltr" class="mw-content-ltr">'''[[phab:T242760|We moved WSExport to a Virtual Private Server (VPS)]]''':  Before this work, the memory and CPU intensive processes of WSExport were too much for the resources on Toolforge. For this reason, we moved the tool to a VPS instance, so that we  could improve overall performance.</span>
# <span lang="en" dir="ltr" class="mw-content-ltr">'''[[phab:T244837|We upgraded Calibre on WSExport VPSs]]''': The VPS was on Calibre 3.48.0, but the latest version was 4.10.1. For this reason, we performed an upgrade, which helped fix various bugs related to generating PDFs. We will also continue with upgrades when new versions are available.</span>
# <span lang="en" dir="ltr" class="mw-content-ltr">We conducted a '''[[phab:T244099|technical analysis]]''' and '''[[phab:T254918|font rendering investigation]]'''. Both of these produced lots of interesting proposals (some of which we’ll share in the section below). In addition, we were able to identify four common types of formatting issues for ebook exports, which are: 1) The original HTML of the wiki, 2) The process of converting the wiki HTML into an ePUB XHTML file, 3) The second output formats (such as PDFs, introduced by Calibre), and 4), The rendering of ePUB files by an eReader. While we can’t do much for the last two (since they involve external software/tools), we can try to improve the situation with the first two.</span>
# <span lang="en" dir="ltr" class="mw-content-ltr">We are currently '''[[phab:T256018|collecting baseline data]]''' on issues related to uptime and errors associated with the WSExport tool, among other data points. This will enable us to compare the original data (i.e., at the beginning of the project) with the data later on (i.e., after changes have been made at the end of the project). This way, we can measure our impact in a meaningful way.</span>
# <span lang="en" dir="ltr" class="mw-content-ltr">We have begun to organize existing Wikisource tickets. We’re currently in the process of '''[[phab:T255512|migrating tickets from GitHub to Phabricator]]''', so they can be consolidated in one place. Once the migration is complete, we propose that we group these tickets into categories, so we have a sense of the general “buckets” of work that we can do. In the future, we ask that all new issues are reported on Phabricator. If you want assistance in how to do this, you can reach out and ask on the project talk page.</span>
# <span lang="en" dir="ltr" class="mw-content-ltr">We have begun to investigate how we can '''improve the user experience''' for all Wikisource users. This investigation is currently in development, and it will be the primary focus of our next status update (which is tentatively scheduled for September).</span>

<div lang="en" dir="ltr" class="mw-content-ltr">
==== Potential next steps ====
</div>

<div lang="en" dir="ltr" class="mw-content-ltr">
===== General reliability =====
</div>

# <span lang="en" dir="ltr" class="mw-content-ltr">'''[[phab:T222936|Investigate cache generated ebooks]]''': With this work, we propose to cache files that are produced, so they do not need to be freshly generated whenever someone wants to download them. This would potentially speed up the process of downloading books. We would need to investigate this proposal first, so we could determine the general impact and scope of the work.</span>
# <span lang="en" dir="ltr" class="mw-content-ltr">'''[[phab:T253283|Investigate job queue for more efficient ebook generation]]''': With this work, we propose implementing a queue-running system. This process would potentially speed up the current download process and give users more information on the download status of a book. One way it could be done is that a user would submit a request for an ebook, which would add it to the queue. The queue would first generate the ePUB, which would then be immediately available for download, and then it would generate the derivative forms (such as, PDF) and make those available when done. We would need to investigate this proposal first, so we could determine the general impact and scope of the work.</span>
# <span lang="en" dir="ltr" class="mw-content-ltr">'''[[phab:T253282|Investigate how to prevent incomplete book downloads]]''': The purpose of this work would be, if possible, to reduce the problem of downloading only a section of a book (when the user expects to download the whole book). We could possibly do this by using subpages from all pages (rather than just from ws-summary) when downloading books. We could also potentially follow subpage redirects, when they exist. We would need to investigate this proposal first, so we could determine the general impact and scope of the work.</span>

<div lang="en" dir="ltr" class="mw-content-ltr">
===== Content rendering =====
</div>

# <span lang="en" dir="ltr" class="mw-content-ltr">Through consulting on the talk page, we have learned that we should also be looking into '''improving support for mathematical equations'''. While we covered [[Special:MyLanguage/Community_Tech/Ebook_Export_Improvement#Example_#2:_Fonts_not_rendered|font rendering issues]] in our original investigation, we didn’t talk about numbers and equations. We’ll look into mathematical equations, as well.</span>
# <span lang="en" dir="ltr" class="mw-content-ltr">We have learned through the font rendering investigation that we can '''[https://github.com/wsexport/tool/pull/230 switch to a new system of fonts]'''. This would enable us to use fonts from a host system (Debian), which has a much larger library of fonts than our own library. For example, this work would give us access to multiple Kannada fonts (while our current font set offers no specific Kannada support).</span>

<div lang="en" dir="ltr" class="mw-content-ltr">
==== Open Questions ====
</div>
''<span lang="en" dir="ltr" class="mw-content-ltr">We would now love to read your feedback, especially in response to the questions below. Remember that we’ll also have a few round of collecting feedback, so we’ll engaging you again in the future as well. You can add your '''[[Talk:Community_Tech/Ebook_Export_Improvement#Early_findings:_Request_for_feedback_(August_2020)|feedback on the project talk page]]'''. Thank you in advance!</span>''

* <span lang="en" dir="ltr" class="mw-content-ltr">What are your general thoughts about the guiding principles that we have learned from the consultation so far (i.e., “Lessons from the consultation”)? Is there anything that you think we should add or change?</span>
* <span lang="en" dir="ltr" class="mw-content-ltr">Is there anything you would like to share about the work we have done so far (i.e., VPS work, Calibre upgrade, various investigations, and the consolidation of tickets)? We’re open to any thoughts or suggestions!</span>
* <span lang="en" dir="ltr" class="mw-content-ltr">What do you think of the proposal to [[phab:T222936|investigate cache generated ebooks]]? Would this be useful and high-priority, in your view? Do you have any concerns?</span>
* <span lang="en" dir="ltr" class="mw-content-ltr">What do you think of the proposal to [[phab:T253283|investigate job queue for more efficient ebook generation]]? Would this be useful and high-priority, in your view? Do you have any concerns?</span>
* <span lang="en" dir="ltr" class="mw-content-ltr">What do you think of the proposal to [[phab:T253282|investigate how to prevent incomplete book downloads]]? Would this be useful and high-priority, in your view? Do you have any concerns?</span>
* <span lang="en" dir="ltr" class="mw-content-ltr">What do you think of the proposal to [https://github.com/wsexport/tool/pull/230 switch to a new system of fonts]? Would this be useful and high-priority, in your view? Do you have any concerns?</span>
* <span lang="en" dir="ltr" class="mw-content-ltr">What work or investigations would you like to see that is *not* being addressed or is being addressed in a different way than you would expect? In other words, what do you think we’re overlooking, if anything?</span>
* <span lang="en" dir="ltr" class="mw-content-ltr">Anything else you would like to add?</span>


[[Category:Wikisource{{#translation:}}]]
[[Category:Book tool{{#translation:}}]]