Arthur Milchior's personal blog - DevelopmentSome thoughts of Arthur Milchior2023-05-17T00:56:21+02:00urn:md5:e37b567a6ae981620691912741ed7239DotclearIt's easier for me to get in Big Techurn:md5:f0bf2d92ed3ab06dbdcca752b67ea3632023-04-07T01:15:00+01:002023-04-07T01:15:00+01:00Arthur MilchiorDevelopment<p>I've heard that it's really hard to get into big-tech companies. That, since
they pay top salaries, they can be very selective in who they hire. I'll plainly
admit that it's not my experience, and offer a different narrative.</p>
<p>And while I admit I believe I understand more or less why it occurs, I still
find it counter-intuitive that it was easier for me to find software engineer job at Amazon and Google<sup>[<a href="http://www.milchior.fr/blog_en/index.php/post/2023/04/07/It-s-easier-for-me-to-get-in-Big-Tech#wiki-footnote-1" id="rev-wiki-footnote-1">1</a>]</sup>, than to join smaller French
companies.</p>
<div class="footnotes"><h4>Note</h4>
<p>[<a href="http://www.milchior.fr/blog_en/index.php/post/2023/04/07/It-s-easier-for-me-to-get-in-Big-Tech#rev-wiki-footnote-1" id="wiki-footnote-1">1</a>] I also heard good feedback from Meta recruiter, up to a sudden email stating that there were no more job in my location. Followed two days later by announcement of mass layoffs.</p></div>
<p>I'll start with a few failed-recruitment stories, and then tries to generalize.
Except that, while writing this post, I realize that this story only generalize
to me, and probably not to most developers that are not fan of theoretical
computer science. However, as I was told to post it anyway, here is the post.</p>
<h3>Pay-related discussion</h3>
<p>First, I want to mention the best rejection I ever got. I was looking to come
back to France. I already had Google and Amazon on my resume. I had a call with
an in-house recruiter of a fast-growing French company. Some well-known national
website, with a good reputation.</p>
<p>The recruiter quickly explained to me that, even if my profile was interesting,
they fear they could not offer me enough money to retain me. So, they decided
not to go forward with my application to avoid making us all lose time. This was
particularly honest and frank, I sincerely appreciate that.</p>
<p>And they were right, I was able to get the double of what they could have
offered. I'm not blaming them, they probably really could not afford my current
compensation, and I'm sure plenty of devs on the marked could fulfill their
need. Worse, since it's not a "startup", they don't pay with stock, options, or
anything fancier than a base comp and annual bonus, they could not even
compensate the low base salary with some vague promise for the future. Still, if
most companies paying an average French salary won't consider my application
because of Google on my resume, that may be a problem if one day I need to find
a job again.</p>
<p>I also had the reverse problem once. The recruiter asked me how much I wanted to
be paid. I gave her my current compensation, and she immediately answered that I
should give myself realistic expectations. This is a mistake I don't do anymore,
and just stop the conversation if I don't get their salary range<sup>[<a href="http://www.milchior.fr/blog_en/index.php/post/2023/04/07/It-s-easier-for-me-to-get-in-Big-Tech#wiki-footnote-1" id="rev-wiki-footnote-1">1</a>]</sup>.</p>
<h3>We require year of professional experience in this particular field</h3>
<p>I often hear recruiters mentioning that it's too bad I don't know such or such
field. I kind of agree, a lot of fields seems fun, and I don't have time to know
them all. Still, I just want to mention one case that was utter most absurd. But
first, a slight bit of context.</p>
<h4>My Background</h4>
<p>I've been contributing to AnkiDroid, an android app with millions of monthly
users, since 2019. My resume, my linked-in profile, state that I'm proficient in
Android development.</p>
<p>I'm certainly not an expert in Android. I don't care about becoming an
expert. I'm never going to know the in and out of every details of the Android
operating systems. But I certainly know enough Android to do the changes that
need to be done to our applications, and to help newcomers grow as contributors
in our project.</p>
<p>More importantly, I can read documentation and usually start working on
whatever needs to be done quickly, even if I'll always be slower than the
mythical 10x engineer that knows everything by heart.</p>
<h4>Home cooking</h4>
<p>I was on a call with a recruiter. They were looking for talents for a
"start-up". The goal of the start-up was to create an app that suggest recipes
to use the food you have in your kitchen$$And if you miss an ingredient, they'd
partner with a fast delivery company so that you can get the missing ingredient
in 15 minutes. That's how they would make money I guess.$$. It's not rocket
surgery, I'm pretty sure I'd have the level required to work on it.</p>
<p>It seems like the recruiter could not agree with my assessment. She explained to
me that they were looking for people with at least a year of professional
experience of Android development. However, as my job never involved Android, we
could not move forward. She could not take open source contributions into
account in the forms HR gave them. The fact that our project had 4.8 stars and 2
millions active users does not matter, I was not getting paid to do it.</p>
<p>Now that I'm not desperate for a job anymore, I find this story very
funny. Because, when I joined Google Chrome for iOS, I had never done any iOS
development. Still, the hiring manager trusted that I could learn the skills
needed, and indeed, I'm now an active contributor and regularly contribute to
Chrome.</p>
<p>The usual explanation is that a young start-up need immediately productive devs,
while a big tech company takes can afford to spend a few month training new
employees. Also, Chrome is huge, so even an experimented dev will need time to
discover the codebase, while a greenfield employee can start contributing
immediately.</p>
<p>Still, this lead to the usual issue: "you need experience to get experience". It
seems like I can change my career in a big tech company, and get the time to
learn and improve my skills. I have no idea how I could join such a kind of
French small company, given that I would need the professional experience in the
specific skills they need before I could get a job.</p>
<h3>Interview question</h3>
<p>Let's now consider the interviews themselves. After all, sometime, I go to the
interview step of the application.</p>
<h4>Android specific questions</h4>
<p>For another Android job, the first question consisted in creating a new Android
app. They provided mock and the app should implements it. I plainly explained to
the interviewer that I have no experience creating new Android app, and I would
need to take some time reading how to do it. Visibly, it has proven to the
interviewer that I'm just incompetent, and the interview quickly ended.</p>
<p>I'm still conflicted about this experience. On the one hand, I since learned
this skill, and I admit it's extremely easy to create a new App in Android and
implement an activity with a view. Today, I could certainly do the exercice in
less than 20 minutes. And if I were creating new apps every month, I could
probably learn to do the exercice in five minutes top. On the other hand, in
which context must you create a new app every month? Yeah, maybe if you are a
freelancer doing small ad-hoc app for various clients, it would make sense. I'm
certain people have this experience. And I feel like a question that excludes
people who worked on big project, spending years on a single application,
ensuring the application can grow over time and adapt to user demand.</p>
<h4>Big tech interview question</h4>
<p>On the other hand, when applying at Google and AWS, I was asked general
algorithmic and system design questions. The kind of very generalist problem
that you learn to solve at university, and that does not require expertise in
the details of any specific program language or developing platform or IDE.</p>
<p>Sadly, my job never rarely consists in solving complex algorithmic problem. Even
when I was doing query-optimization software, it was a very small part of the
job. So, I must admit that those questions are even less relevant to my
day-to-day job than "creating a new android app".</p>
<p>Even during my time at AWS, when I was working on a software to optimize SQL
query, it was rare that I actually had to consider complex algorithmic
question.</p>
<h4>Comparing the questions</h4>
<p>I used to be a searcher, I created algorithm during my PhD and
post-doc. Algorithmic questions are simple to me, and it's something I actually
enjoy to read about for fun. On the other hand, I never cared about learning the
intimate details of any system, any library. Paradoxically, not being an expert
means that it was easier to get a big-tech job than a small-company job.</p>
<div class="footnotes"><h4>Note</h4>
<p>[<a href="http://www.milchior.fr/blog_en/index.php/post/2023/04/07/It-s-easier-for-me-to-get-in-Big-Tech#rev-wiki-footnote-1" id="wiki-footnote-1">1</a>] unless they seems to be a very exciting opportunity.</p></div>
http://www.milchior.fr/blog_en/index.php/post/2023/04/07/It-s-easier-for-me-to-get-in-Big-Tech#comment-formhttp://www.milchior.fr/blog_en/index.php/feed/atom/comments/786Kotlin migration of AnkiDroidurn:md5:a19466a592e449a96a7c30a043f9888f2022-09-23T23:45:00+01:002022-09-23T23:45:00+01:00Arthur MilchiorDevelopment<p>AnkiDroid successfully converted its entire project from Java to Kotlin. In this first AnkiDroid dev blog post, we’ll explain our goal, our methods, and what remains to do.</p>
<h4>Our goals</h4>
<p>The main goal of the migration is to improve developers' experience. Kotlin code is easier to read, write and review than Java. Anki AnkiDroid is 13 years old. If we are still here in 13 years, we hope that the time spent during this migration year will be entirely saved. We also hopefully will decrease the number of bugs and get more readable codes using Kotlin features, such as better nullability check, scope function, and high-order function over collections.</p>
<p>Here is the extra goals we discovered along the way:</p>
<ul>
<li>git history should still allow us to track the origin of each code line.</li>
<li>Each commit should compile</li>
<li>New contributors should be able to help us easily</li>
<li>Kotlin migration should not introduce bug</li>
</ul> <h4>How we achieved them</h4>
<h5>Git history</h5>
<p>By default, if you just migrate a file foo.java to foo.kt, the file changed a lot, and git believe a file was deleted and a file was added. Instead, we renamed the file to foo.kt in the first commit, and converted its content to actual Kotlin code in the second commit.</p>
<p>This way, git history sees a rename and then a file change. Hence, if a line of code is surprising and we need to understand why it was introduced in the codebase, we could look at git history, for example through `gitk`, find the commit that did the migration, then find the original java line, and finally find where this java line was introduced.</p>
<h5>Each commit compiles and tests passes</h5>
<p>Being able to compile at each commit is important for `git bisect`<sup>[<a href="http://www.milchior.fr/blog_en/index.php/post/2022/09/23/Kotlin-migration-of-AnkiDroid#wiki-footnote-1" id="rev-wiki-footnote-1">1</a>]</sup>.</p>
<p>Since we had a java file called `foo.kt`, we needed to let gradle, the tool used to build AnkiDroid, know that it <a href="https://github.com/ankidroid/Anki-Android/blob/ee1a473fa105326dd5739a8fd9eeb61561524506/AnkiDroid/kotlinMigration.gradle">must consider</a> that this file is actually a java file. This was essentially done by changing compilation in the following way:</p>
<ul>
<li>Ignoring foo.kt</li>
<li>Copying foo.kt to foo.java</li>
<li>Actually compiling</li>
<li>Deleting foo.java</li>
</ul>
<p>As far as I can tell, this migration tool was the work of David-Allison, MikeHardy and Arnold2381<sup>[<a href="http://www.milchior.fr/blog_en/index.php/post/2022/09/23/Kotlin-migration-of-AnkiDroid#wiki-footnote-2" id="rev-wiki-footnote-2">2</a>]</sup>.</p>
<p>This gradle file also has to contain explicitly what was the actual value of `foo.java`, and whether this file was part of the production code, the unit test code or the android test code.</p>
<h5>Minimizing the risk of bug</h5>
<p>As should be obvious to any developer that worked with both Java and Kotlin, a java file migrated to Kotlin is not actually a “Kotlin file”. Same as a French text automatically translated to English would not be proper English, but still be relatively useful and understandable.</p>
<p>Thas is, translating to Kotlin would require a lot of extra manual work.</p>
<p>We could require it to be done at the same time as the migration. We decided against it because,</p>
<ul>
<li>This would take a lot of extra time, making the migration even longer</li>
<li>It is hard to check that the code change do not change the behavior when you can only compare the change to java code,</li>
<li>Even if you did the change in a third commit, you would still risk to disagree during review, which would block from merging the migration itself,</li>
<li>Some rewrites would require to touch multiple files simultaneously, which makes the review harder and the risk of merge conflict greater.</li>
</ul>
<p>Instead, we simply added a `@KotlinCleanup` annotation, that would indicate what needs to be done. The details of the migration can always be discussed in individual PR later.</p>
<h5>Ensuring single-file migration is easy</h5>
<p>IntelliJ has a really great tool that converts a java file to kotlin almost flawlessly<sup>[<a href="http://www.milchior.fr/blog_en/index.php/post/2022/09/23/Kotlin-migration-of-AnkiDroid#wiki-footnote-3" id="rev-wiki-footnote-3">3</a>]</sup>. However, the changes to the build file and the two commit rules makes the process of submitting a PR long. Especially when it had to be done 861 times and 20<sup>[<a href="http://www.milchior.fr/blog_en/index.php/post/2022/09/23/Kotlin-migration-of-AnkiDroid#wiki-footnote-4" id="rev-wiki-footnote-4">4</a>]</sup> people<sup>[<a href="http://www.milchior.fr/blog_en/index.php/post/2022/09/23/Kotlin-migration-of-AnkiDroid#wiki-footnote-5" id="rev-wiki-footnote-5">5</a>]</sup> had to understand and apply the process. So Arthur-Milchior introduced a <a href="https://github.com/ankidroid/Anki-Android/blob/9a8ae65c2d01439daae8802a8cd5b08f8b77b4c1/tools/migrate.sh">migration</a> shell script that Codingtosh ported to Mac and generally improved.</p>
<p>This way, any contributors only had to use the IntelliJ tool to convert a file, ensure the test passes, and run this script to create the two commits for the pull request.</p>
<p>Also, the KotlinCleanup annotation ensures that any new developer that wants to start contributing with us can just look for KotlinCleanup annotation, and apply them, letting simple tasks for them to start.</p>
<h4>After the migration</h4>
<p>Here are a few changes the migration allowed to us. A lot of them could actually be started before the migration was complete. However, waiting until the end of the migration ensured that we could do it in a single change instead of having to redo it regularly when new files were converted.</p>
<h5>Introducing coroutine.</h5>
<p>This work is still in progress. The migration from AsyncTask, that Ankidroid used previously, is Divyansh’s <a href="https://docs.google.com/document/d/1svTqQn6vCBEDhfFDz1h6Oe9uzVn5Hi8vmFB64OrYETo/edit">2022 Google Summer of Code</a> project. We also plan to use coroutine for all access to <a href="https://github.com/ankidroid/Anki-Android/wiki/Accessing-the-collection">our data</a>, which was not yet the case.</p>
<h5>Using Kotlin data structures.</h5>
<p>In particular, HashSet/Map and ArrayList are the same as java, but with better typing around nullability. Also, in a lot of case, using map, filter, forEach, is really nice. `listOf` and other functions that creates collections from a set of explicit elements are great too.</p>
<p>Also, formatted strings are great in terms of readability.</p>
<h5>Update Material Dialogs library</h5>
<p>Newer versions of the <a href="https://github.com/afollestad/material-dialogs">library</a> didn’t support Java anymore.</p>
<h4>Conclusion</h4>
<p>This is a very personal conclusion by Arthur Milchior - main author of this post, but certainly not of the migration.</p>
<p>I find this process is nothing short of amazing. Be it on the technical or on the human side.
On the technical side, JetBrains was able to create a language that is greater than Java to code with, and still entirely compatible, ensuring a smooth migration. On top of that, we edited our build tool to ensure that a .kt files would be compiled as java to improve the migration. And if that was not already meta enough, we also created a script that would edit the build tool automatically.</p>
<p>Humanly, it’s even more impressive. You have a bunch of 20 people across the world, who never met, and who collaborated on a goal to help potentially a hundred or two fellow developers in the future years (or decade ?), improving their experience, so that they could help the millions of users relying on AnkiDroid.</p>
<div class="footnotes"><h4>Notes</h4>
<p>[<a href="http://www.milchior.fr/blog_en/index.php/post/2022/09/23/Kotlin-migration-of-AnkiDroid#rev-wiki-footnote-1" id="wiki-footnote-1">1</a>] Git bisect allows us to give it a list of commits to ignore, but we should compile this list and this is extra complexity on an already complex debugging process.</p>
<p>[<a href="http://www.milchior.fr/blog_en/index.php/post/2022/09/23/Kotlin-migration-of-AnkiDroid#rev-wiki-footnote-2" id="wiki-footnote-2">2</a>] Checking for the authors of this file is harder than any other file, since each migration commit had to two this file to add the name foo.java. So I could not trust github statement that this file had 19 contributors.</p>
<p>[<a href="http://www.milchior.fr/blog_en/index.php/post/2022/09/23/Kotlin-migration-of-AnkiDroid#rev-wiki-footnote-3" id="wiki-footnote-3">3</a>] The only exception I encountered was with the String’s split method. In Kotlin, by default, split remove empty string starting and ending a split. While in Java, those empty strings are kept. JetBrains answered it was corrected in a version of the tool that I didn’t had yet installed.</p>
<p>[<a href="http://www.milchior.fr/blog_en/index.php/post/2022/09/23/Kotlin-migration-of-AnkiDroid#rev-wiki-footnote-4" id="wiki-footnote-4">4</a>] According to `git log --format='%aN %s' |egrep -i "to (kotlin|.kt)"|awk '{print $1}' |sort -u |wc -l`</p>
<p>[<a href="http://www.milchior.fr/blog_en/index.php/post/2022/09/23/Kotlin-migration-of-AnkiDroid#rev-wiki-footnote-5" id="wiki-footnote-5">5</a>] Arthur, Brayan, codingtosh, Damien, David, Divyansh, dorrin-sot, Kilian, lukstbit, MacAndSwiss, Mani, Nishant, oyeraghib, Piyush, Prateek, Prince, puranjayK, Shridhar and Timo.</p></div>
http://www.milchior.fr/blog_en/index.php/post/2022/09/23/Kotlin-migration-of-AnkiDroid#comment-formhttp://www.milchior.fr/blog_en/index.php/feed/atom/comments/783How to find the part of the code which implements some feature of a softwareurn:md5:aaa2048794feed202c4277847ff7ae552022-04-03T03:29:00+01:002022-04-03T03:29:00+01:00Arthur MilchiorDevelopment<p>In this article, I intend to give all tricks I know to figure out which part of the code is in charge of a behavior in a program. This assumes that you don't already know the codebase and don't know where to look for. I'll go by what I consider to be the easiest to the hardest.</p> <h3>The tools I know</h3>
<h4>Looking for the answers in the logs</h4>
<p>Assuming your software has logs and you can access them, they may simply state the name of the function executed, or the name of the file+line numbers.</p>
<h4>Analyzing the running app</h4>
<p>This assumes that:</p>
<ul>
<li>you can compile the code,</li>
<li>you can run the code (on the computer which compiles, an emulator, a connected device) and</li>
<li>you have a tool that analyze the current state of the application.</li>
</ul>
<p>Such a tool can be a profiler. With graphical program, it can be a tool that explode the view in part and allow you to see what are each part of the view. For text interface, it can be a debugger connected to a program running on another thread, and showing the current stack trace.</p>
<p>If you don't know how to use such tools, it may be really useful for you to learn how to use them; the search will be quite slower the first time, but you'll win incredible time later. However, those tools are sometime really hard to set-up. For example, if you run microservices, if you run code in multiple languages, if you have separated applications dealing with background, foreground... In AnkiDroid, the code I know the best, the main software is in Kotlin, but some backend is done in Rust, connected through protobuf messages, and some UI is displayed using webviews; I plainly admit I would not know how to profile this back-end and this view.</p>
<h4>Looking for strings</h4>
<p>Strings can appear in the UI or in the logs. You can look for a string, or at least a string template, that seems to be used mainly in the part of the software you want to consider. Then you can find where this string is used in the software. This may not be exactly the part of the code you were looking for, because a lot of the time, that will lead you to the view, and you may be more interested in a model or a controller (assuming the MVC model); however, it may be case that the controller is in the same folder as the view. Or that at least you can find who creates this view and what controllers/models is used together.</p>
<h4>Asking someone who knows the codebase</h4>
<p>The easiest method is to ask someone who knows the codebase. In my experience, if you are able to ask a clear question "what function is in charge of the action triggered by this button of that view", people are keen to answer. Indeed, if someone knows the answer, it's very easy to transmit, you just give a function name/class name/file name and line number.</p>
<p>However, I discovered this to fail in two oppositive case. If the original author resigned, or the code repository is not really active anymore, everybody forgot the codebase. On the contrary, if the codebase is extremely active, it may be hard to figure out who is the relevant person. Worst, I did a few contribution to cpython documentation and to some of its tools, I found it quite difficult to find people who can help understanding cpython code source, because most of the volunteer who spent time helping people where here to help people learning python; and it was hard to convince them I already am profficient in Python and that I was not here to write code in Python but to write code FOR Python. (Actually, some of the documentation tooling is in Python, so I was also here to write code in Python, but that was not relevant.)</p>
<h4>Fuzzy searching</h4>
<p>Most modern IDE allow you to do fuzzy search for class/file/function name. Most of the time, names in codes are similar to names in the English version of the running software. So you can just search and see what you get. That may require to explore multiple search result, because it's not always easy to figure out which one is relevant.</p>
<h4>Adding breakpoints to library functions that must be used</h4>
<p>Usually, one debugs with a debugger; however, the debugger is almost useless if you don't know the code base as you can't put break-point on the code you want to debug. However, you can also add breakpoints to libraries used by the code you want to debug. Let's say for example that you know that the action you want to discover send a very generic log such as "download completed". You can't look at all the places that log "download completed" because your "download" function is called everywhere. However, you can put a break-point on the logging function that gets triggered only if the message contains the word "download"<sup>[<a href="http://www.milchior.fr/blog_en/index.php/post/2022/04/03/How-to-find-the-part-of-the-code-which-implements-some-feature-of-a-software#wiki-footnote-1" id="rev-wiki-footnote-1">1</a>]</sup>. You deactivate all breakpoints until you are almost doing the action you want to understand. Then you reactivate the breakpoints. And then you run the software up until the log occurs. You can then look at the stack-trace. If you didn't find the function you were looking for in the stack-trace, maybe you need to resume the run, because you may log "download" multiple time. Eventually, you'll have found the method you want.</p>
<p>Beware that on multi-threaded software, the function you are looking for may not be in the stack trace because it would be on a different thread. If you're lucky, this thread is waiting for the end of the download, and so if you look at other thread's stack-frame, you'll find the function you are looking for.</p>
<h3>I'm certainly forgetting things</h3>
<p>It's almost certain there are tricks I don't know. I'll be glad if you can alleviate my ignorance.</p>
<div class="footnotes"><h4>Note</h4>
<p>[<a href="http://www.milchior.fr/blog_en/index.php/post/2022/04/03/How-to-find-the-part-of-the-code-which-implements-some-feature-of-a-software#rev-wiki-footnote-1" id="wiki-footnote-1">1</a>] At worst, you modify the log function such that you add a code-path to log "download" specifically. And you put the break-point in this code path.</p></div>
http://www.milchior.fr/blog_en/index.php/post/2022/04/03/How-to-find-the-part-of-the-code-which-implements-some-feature-of-a-software#comment-formhttp://www.milchior.fr/blog_en/index.php/feed/atom/comments/778