The story up ’till now
You might remember some earlier posts I’ve made about Fedora package social networking. The background here is that we built a web application called Fedora Community a couple of years ago. With Luya Tshimbalanga’s help, I ran a number of usability tests on this initial version at FUDcon Toronto. The results pointed out some key problems that over the past couple of years have definitely negatively impacted user adoption, including slow search and loading times.
Late this summer Luke Macken, John Palmieri, Spot, and myself went through those all of those issues and formulated a plan to try again and build a better application for package maintainers. We launched a beta version at FUDcon Blacksburg this past Saturday, and it’s called Fedora Packages.
We wanted to build only the most core functionality and keep things very light and speedy, being overly cautious towards adding features beyond the initial core to avoid the problems we ran into with the first version. I want to emphasize this because a lot of you provided great suggestions for functionality but we decided to focus just on the core functionality for now and have planned to work on several of your suggestions in future releases.
Screenshot, front page of Inkscape’s profile
You can think of Fedora Packages like a social networking system, where instead of profiles of people, you’ll find profiles of packages. We load the package’s icon in the upper left. Right now, we just load the icon if we have it, and if it’s too low-resolution, we blow it up. For some packages like clonekeen, this looks pretty awesome but for others, not so much. We very much appreciated your suggestions on an earlier blog post about package categories and how to display icons for packages that don’t have icons or high-quality icons, but again in keeping with our focus on the core and having something ready for you to check out by FUDcon Blacksburg, we decided to shelve that for now and focus on higher-priority features like optimizing our searches.
We also display basic details in the left sidebar such as like the latest version of the package in Fedora, the package owner, any packages in its family tree, and these pieces of information persist across the package’s profile tabs.
The front overview page shows the description of the package, what versions of the package are in what Fedora and/or EPEL releases, and a link to the upstream. I am hoping that in the next cut we can work with Vincent and the OpenSuSE folks on extending our upstream metadata; Luke has some ideas on this as well (more on that later.)
One core piece of functionality we definitely heard loud and clear from your feedback on Fedora Community, including during the FUDcon Toronto usability tests, was that the Fedora Community URL structure was complicated and made it difficult to go directly to the page you wanted. With Fedora packages, you need only append the name of the package you are interested in after the /packages/ portion of the URL and it will take you straight there. You can go directly to a particular tab of the package profile by appending the name of the tab after the package name portion of the URL (for example, https://community.dev.fedoraproject.org/packages/inkscape/bugs to go to the Inkscape bugs tab.)
Some new goodies: Patch Viewer and Contents Viewer
Fedora Packages’ patch viewer
You may recognize some of the tabs in Fedora Packages – both the builds and updates tabs are widgets Luke and J5 had built for Fedora Community. They have been cleaned up and optimized for inclusion here. We put some new goodies for you to work with as well.
Luke worked with me on the patch viewer, which not only gives you a full listing of all Fedora patches against any given patch with the committer and commit date, but also lets you view a diffstat per patch as well as across all patches for the package. You can also view the full content of the patches with code syntax highlighting inline in the page.
Expanded view on a patch from Fedora Packages’ patch viewer
J5 worked on the package contents tab, which displays the contents of the RPM binary for the package (see screenshot below.)
There are of course more nooks and crannies of useful information in each package profile: please try Fedora Packages out and let us know what you think!
Inkscape’s contents tab in Fedora Packages
Optimized search via playing games: Fedora Tagger
Fedora Packages search results for ’email’
So how did we get our search to work so quickly this time, and how are we getting better search results?
Luke did a lot of research on how to best improve our search and concluded that the GPL-licensed Xapian Open Source Search Engine Library was the way to do it. Xapian searches package names, summaries, descriptions, and tags, and it shold weight the search results in the following manner (although John is still tweaking it, and I’m not 100% sure where tags are in the ordering):
- If the search term(s) appear in the package name.
- If the search term(s) appear in the package tags.
- If the search term(s) appear in the package summary.
- If the search term(s) appear in the package description.
“Where do the tags come from, though?” you might ask. Good question! 🙂 Fedora PackageDB has a tagging system and yum is able to access those tags, so Ralph Bean pre-populated the Xapian index with the tags from Fedora PackageDB. Not all packages in PackageDB have tags, though. What to do?
Other features include a naughty-words filter (no, Tagger is not a dumping ground for your drama 🙂 ) and anonymous tag voting, although you can’t add tags if you are anonymous. Log in and try Tagger, and let us know what you think!
Planning whiteboard for Fedora Packages and Fedora Tagger
Where we go from here
Our FUDcon presentation, “Making Fedora Package Maintenance Easier”, was (thankfully! phew!) given to a completely full room and the initial reception seemed to be very positive. Here’s a rundown of the comments and feedback we got during the session from those that attended:
Colin explaining his upstream package metadata ideas
- Would it be possible to show which patches were sent upstream and are pending, which were refused by upstream, and which have not yet been submitted? (Colin Walters & Nathaniel McCallum)
- One proposed solution during the discussion was to create additional rules/regulations for package maintainers to note this information in their package commits. This was noted to be an extra burden on maintainers, though.
- Another proposed solution that Luke has been looking into is called DOAP files.
- Colin noted that he was upset when people fix bugs downstream in code he is the upstream for and don’t even let him know about it. He ends up putting time into fixing bugs that have already been fixed because he just didn’t know.
- Both Nathaniel and Casey Dahlin noted that there is some functionality available in git where it can tell you if your lines of code from a patch is available in a given repo.
- Nathaniel suggested adding something to the DOAP files, such that whenever someone builds a patch and commits it, it would automatically notify upstream. Spot noted that there are quite a few patches that are not suitable for upstream and wouldn’t be useful to them because they are Fedora-specific. He suggested noting the upstream contact on the page of each package. Ben Boeckel suggested that maybe Fedora-specific packages could start with numbers in the 1,000 range, but Spot thought the kernel maintainers wouldn’t like that very much.
- Ben also mentioned an upstream version checker called ‘fever.’ Spot noted that Luke has been investigating its inclusion and it’s now called cnucnu. (More details on the Fedora wiki.)
- Cool, this is like extensions.gnome.org for Fedora! (Paul Frields)
- It’s not a store, though (Spot)
- With a few small tweaks–filtering down to apps only, and adding a install button–we could build a separate app like that using this codebase, though. (J5)
- What about screenshots? (Rahul Sundaram)
- We don’t have auth on the site, so we couldn’t let users upload screenshots. (Spot)
- Maybe we could set up a separate service to allow that. (J5)
- Can yum use the tags? Can other applications use the tags? (Kevin Fenzi)
- Yum already uses tags from PackageDB … (lmacken)
- Maybe we could get rid of the extraneous pkgdb stuff so we don’t have duplicate / separate functionality (Toshio)
- What about using the tags in the package spec files? (Colin)
- We could take the tag cloud that we generate… if the tag achieves a certain amount of weight, we could take those tags with those mappings and use in the GNOME shell search logistics.
- Is there a JSON API for this? (Casey Dahlin)
- It’s definitely do-able since it’s built using TurboGears (lmacken)
- Actually, I think it’s not documented yet but you could go to the connectors directory directly. (J5)
- Could we use this to better document specs? (Toshio)
- Toshio had the idea of looking at adding the ability to annotate spec files. There was a project proposal to do this through the Google Summer of Code; the Django project documentation does it now, where you can comment on specific chunks/parts of the documentation rather than leave comments in a generic area. What if we added that ability to specs, so maintainers could document their specs when asked questions to new packagers they were mentoring? We could have a toggle to turn them on or off. The challenge is that we don’t have auth in Fedora Packages – maybe we could get around that by having the maintainer receive patches to the spec file with annotation via bugzilla?
- Full package description text available in tagger – We noticed while playing Tagger that there were many packages we had never heard of and we opened up new browser tabs to do research on them in Fedora Packager. It would be more streamlined to display the package description and upstream link right in Tagger (upon request, after clicking ‘more details’) to save time and effort.
- Mozilla Badge Support – Mozilla’s Open Badges Project is something we could deploy to reward folks submitting tags and voting on tags in Tagger, as well as a way of rewarding our package maintainers and other Fedora contributors.
- PackageKit integration – We could provide an API for PackageKit to use Fedora Packages and its optimized search results for returning package data rather than querying yum for this information. One advantage to this approach is that PackageKit could get the icons from Fedora Packages, even for packages not already installed on the system.
- GNOME Shell integration – As mentioned earlier, the idea came up to allow GNOME shell to access the optimized search including tags to display appropraite applications directly in the shell.
- A query language – We could implement a query language for the search to make it even more efficient to find what you are looking for.
- Subpackage display in search results – This is a minor bug, but right now if your search terms match a subpackage, we display both the subpackage, its parent, and all of its sublings. We should only be displaying the subpackage(s) that match the search terms and its parent.
- Newly-added tags don’t display on the package card right away in tagger – another minor bug we can address.
- Improved linkage to bugs mentioned in spec files and patches – we have some links working now, but folks aren’t incredibly consistent with how they refer to bug numbers in their commit messages and comments so we missed a few.
- Diffs between versions for spec files – is this something you’d be interested in seeing?
- Diffs between the spec and its patches – this could show if you’re carrying patches for a package that aren’t even referenced in your spec (I think we saw a few instances of this too.
- Interaction with the Fedora system accessing Fedora Packages – we talked about possibly having an icon or notification on a package’s profile if we detected that you have it installed, or if you have it installed and there’s an update available for your version. (In the released versions table we could even highlight your release.) We could also pre-populate the release dropdown filters across the UI with your current version. For updates, it might be cool to show which ones you have installed and allow you to easily given them karma.
- Default or not? – some kind of indication on a package profile to show whether or not it is installed by default.
- Timestamp the screens – we could display a timestamp on each page of packages… since we’re working from cached data.
- Subpackage versions bug – right now we give subpackages the same version number as the parent even though they might not have the same version number as their parent, in active releases overview, might fall over in some edge cases on the relationships / contents tabs
- Off-site upstream summary link indicator – this is a super-minor designery desire here, but I thought it would be nice to add an ‘offsite’ icon to indicate the upstream webpage links are outside of Fedora infrastructure (see “Patterns for Expressing Expansion Link Weights in Web Applications” and “More Link Treatments”.
- Maintainer mail-to links – Right now we don’t link the maintainer’s name, it’s just plain text. Pingou suggested that we use the maintainer alias email addresses. We also discussed potentially providing, in a panel that pops up when you hover over the maintainer’s name, showing their IRC nick, emailaddress, and buglist. We opted to not do anything yet because we wanted to see what you might ask for here. What do you think?
- Alpha-sort subpackages list – we don’t do this now I don’t think.
- [NEW] Type-ahead in the searchbox – (idea from CodeBlock) this would be especially awesome once you’re already at a package profile page because you won’t have to bounce to the front page, you can go directly to the page you want
Summary & Where the Action is At
So we hopefully have provided Fedora package maintainers (and users? Maybe?) a better tool for managing information about packages this time. We now also have Tagger, which allows those of us (like me! 🙂 ) who are less technical but still want to help Fedora a way to contribute and learn more about what’s available in Fedora along the way.
My life is now complete, and Tagger made it happen: I discovered xteddy!
I do think it was the right call to keep things as minimal as possible first. I think it left the slate open for a huge number of ideas (as you can see above) directly from you. We still have a lot further to go, though: your help, suggestions, and feedback all this time have of course been critical and we’ll need more. 🙂 If you’d like to get involved, here’s the rundown:
|IRC||#fedora-admin on irc.freenode.net|
|Mailing list||Fedora infrastructure list|
|IRC||#fedora-admin on irc.freenode.net|
|Mailing list||Fedora infrastructure list|
Oh, what about that weird radioactive panda post?
Remember that weird hot-dog-seemingly-with-a-grudge-with-a-panda post? Well, check this out (it will get even cooler soon with Fotios’ parallax animation… heh, heh, heh.)