What I Want to Hear About this Tuesday at the State of the Union Address

I received an email from BarackObama.com asking me to fill out a one question survey.

The survey question was:

What issue are you most excited to hear about in the State of the Union?

This was my response:

The biggest issue that lost my enthusiasm in the leadership of our president is how much the NSA has been sabotaging the security of the internet.

I understand that the President worries about our safety, and that the NSA is telling him that they are making things safer.

Frankly, I don’t believe that it is making us safer, it’s eroding the clear leadership that the US has taken in moving the world forward technologically, and is threatening jobs by undermining the integrity of US tech companies. It upset me greatly that the President focused mainly on phone record meta, who uses the phone these days?


    Introducing Assets Manager for WordPress

    Note: if the links aren’t working properly, resave the pretty permalinks settings.

    Download

    Many of the companies which my current place of employment interacts with have a higher level of security on their firewall (they also tend to use IE7, such is life). Because of this we were having issues sharing files with our constituents using the current industry file sharing tools.

    To solve this problem I was tasked with creating a custom version of the corporate file sharing webapps for internal use. This would solve the problems we were having. All the links would be hosted on our domain, so we wouldn’t have to worry about getting third parties’ domains whitelisted in other company’s firewalls.

    I decided that WordPress would be the best tool to build this on. It already has wonderful custom post management abilities as well as built-in media management tools.

    I’m proud of what I built, so I got permission to release it to the WordPress community as a white-labeled plugin. Special thanks to @binmind for his extensive QA testing of the company’s plugin, his testing was crucial for development of the proof of concept and making sure everything was working as it should.

    Instead of releasing the plugin as-is,  I decided to rebuild it from scratch. I’ve learnt a lot since building the original assets manager  and wanted to harden up the code base before releasing it to the public. Here are the results of my efforts.

    Features

    features

    Path Obfuscation:

    When a file is uploaded to WordPress you usually access it by linking directly to the location of where the file is hosted on the server. Assets Manager creates a unique obfuscated link for the file instead. When a file is downloaded it will receive the name you supply.

    This does two things:

    1. You can’t figure out where the file is actually hosted, nor can you find other files based on some pattern. This is a security feature. Since the links to the files do not indicate anything about where the files are, or what they will be called when downloaded, you can’t guess where other files are stored.
    2. Files are never linked to, they are read and served. This allows #1 to work. It also means that before the file is served, Assets Manager can check various things, like if the user is logged in or if the file has “expired”.

    When should this file expire?

    Because of #2 above, Assets Manager intercepts files before they are served to the user from the server. This means that you can decide when and how the file will be served. I’ve included the ability to set how long the file should last. If you see you’re running out of time, you can extend the expiration by as long as you wish. The expiration date of the file is displayed next to the expiration feature letting you know when the file will expire.

    Enable this file?

    Same as the above feature. If you send out the wrong link, you can easily edit the settings and uncheck “Enabled”.

    Secure this file?

    I can also  check to see if a user is logged in before serving them the file. It doesn’t actually make the file secure. If someone downloads it, they can send it anywhere. It only secures the link to the file.

    Remove file

    When a file is removed it is not deleted, it can still be found in the media library. It is just detached from that assets set. You can delete it via the media library if you wish.

    Stats

    A basic hit count is recorded per file.

    Asset Set

    Each asset set is a custom post type, the upload files are attached to this post. The URL for the asset set is obfuscated to protect it’s location. If it is linked to it will be indexed though. But bots can find it crawling the site.

    You can upload a set of files, then only share the one link. That way if you decide to change the links around you can. Only available files will be listed there. So if a file is “secure” and the user isn’t logged in, they won’t see it, nor will anyone see expired and disabled files.

    Future features I’m working on:

    • Sha1: If you upload a file that already exists it will link that file to your post instead of keeping multiple versions of the file. I believe that WordPress should work this way in general, all filesystems for that matter. That’s a benefit of networks. Why keep doubles, unless you intentionally are backing up the information?
    • File replacement: After uploading and even sharing a file you’ll be able to replace the file behind the active link with a file of the same MIME type. This way if you make a typo you can fix it quickly and replace the file without sending out a new link.
    What do you think?
     If you have ideas, discover bugs, let me know.

      Code Is Poetry

      codeispoetry

      At the bottom of every page of wordpress.org is the above statement, and it’s not just an empty phrase.

      I learned what I know from digging into WordPress. It started by my breaking the site I was supposed to be managing, sorry Karin. Many books, themes, plugins and years later I seem to be able to manage most any PHP site quite proficiently.

      No matter what I’m working on, I try to keep the above in mind. “Code Is Poetry.” If I can make a method more elegant, concise, I go for it.

      Having influenced me so much, I decided to put WordPress to a test. See if the good people at WordPress hold to their own mantra.

      To do so I installed the top CMS platforms on a local environment so I could compare their codebases and database structures with each other. I wasn’t very scientific about what is considered a “top” CMS. I pretty much Googled and made a list of the top few that came up the most. I have not run any performance tests, I may do that for another post. This post is just about structure of code and database. “Code is Poetry” right? Here are my results.

      cms file search

      File count (CMS’ in alphabetical order)
      Concrete5: 4006 files
      Drupal: 1065 files
      Joomla: 5083 files
      WordPress: 1062 files

      cms folder search

       Folder count
      Concrete5: 765
      Drupal: 136
      Joomla: 1233
      WordPress: 112

      Top level folders
      Concrete5: 20
      Drupal: 7
      Joomla: 17
      WordPress: 3

      Why This is Important

      A codebase to a developer is a lot like moving parts in electronics. There more there is, the more that can break. Less doesn’t necessarily mean better, a space shuttle is clearly better than a 747 and has far more moving parts. But to continue the analogy, a SSD is far superior to a HDD.

      Drupal and WordPress are neck and neck in numbers, though, WordPress is ahead by a hair ahead, except for the top level folder stat.

      The top level folder stat is important. WordPress wins hands-down here. Aside from having strong OCD tendencies, it’s important because it’s an indication of the overall clarity of structure of the codebase, which has clear ramifications. Try upgrading WordPress, one click. Try upgrading Drupal… HA!

      The WordPress codebase is structured beautifully with clear delineation between wp-includes, wp-admin, wp-content. It’s clear what is where, and what is what. You do not have to read through their documentation to see clearly where the core sits, and where you can mess around. You cannot say this about the other CMS platforms.

      cms folder breakdown

      Now for the Databases: Table count
      Concrete5: 172
      Drupal: 72
      Joomla: 68
      WordPress: 11

      For more about the elegance of WordPress’ database read: How WordPress Works: Dissecting the Database.

      In conclusion, I don’t want, ever again, to hear about how bloated WordPress is.


        How WordPress Works: Dissecting the Database

        The WordPress Database

        There is beauty in the simplicity of WordPress’ database structure. All the functionality of posts, pages, custom posts, taxonomy, users and core settings are here. In 11 tables.

        For comparison, the almighty Drupal has 72 tables, Joomla has 68.

        All posts, pages and custom posts are saved in the `wp_posts` table. They are differentiated by the `post_type` column. Any additional data you need to save with your post (whatever the post_type is) can be stored in `wp_postmeta`.

        Metas are extremely powerful. You can extend everything in pretty much any way with them.

        Example: Your site manages the courses of an educational institute. So you create the post_types of ‘Course’ and ‘Lecturer’. Now you can save in the `post_content` all about the ‘Course’ and ‘Lecturer’, but what if you need to store extra information about each, that you’ll need to access easily. For a course you might want to know the dates the course is taking place. If you save that in the ‘post_content’, as part of the other descriptive content, you will not be able to run queries easily on that information, you can’t sort it, pull it out for widgets etc. That’s where meta comes in.

        wp_postmeta table

        Each of the meta tables, postmeta, commentmeta and usermeta each have 4 columns: meta_id, post_id (or the equivalent), meta_key, and meta_value. Each post can have whatever extra meta you need, and it can be pulled out with a simple SELECT WHERE meta_key = ‘X’; command.

        And that’s pretty much it. All of WordPress’s functionality is there. Comments, users, and posts all have their basic structure in their main table and all can be extended as much as needed through their meta.

        Taxonomy is somewhat more complicated. It requires 3 tables. wp_term_taxonomy stores the types of taxonomies. Categories, Tags, and any other custom taxonomy type you create will be here. The individual terms will be in wp_terms. So if you have 3 categories and 15 tags in your site, each of those will be stored in wp_terms. wp_term_relationships links them all together keeping it all in order. Easy-peasy, right?

        The basic options of the WordPress install are in wp_options. The only table out of order is wp_links, a relic of installs past. Today all the link functionality can easily be incorporated as a custom_post_type. But because WordPress cares about backwards compatibility, the table remains.

        That’s it. Lean and mean.

        One question that comes up about meta is, doesn’t that mean that there are a lot of extra queries hitting the database? This would be true, if not for the caching system of WordPress. So each time you call get_post_meta() you’re not hitting the database. So you’re good.

        So when people say that WordPress is “bloated” I’m not quite sure what they’re talking about.


          WordPress Proposal: “Deep” Linking Taxonomies to Custom Posts

          EDIT: A very awesome plugin that does this and much more, exists. Go check out Piklist.

          Scenario

          You are building a site for an educational institute. There are several requirements:

          • Speakers – These are the people giving the courses. There could be different speakers for the same course, if there are too many students for one course, or on different years.
          • Courses – Each course could be unique, or it could be the same required course that every student needs to take to get through.
          • Dates – The duration. If you’re dealing with conferences, it could be a single date. If it’s a course, it may be a time-frame.

          Each of these could and should be a custom post type. And each would have its own custom taxonomy. Speakers should have a Department taxonomy. Courses should as well. Dates should have a Semester taxonomy.

          Here’s where things get interesting. What if a Speaker had a taxonomy of Course, so all the lecturers of a specific Course could link themselves to that Course? Wouldn’t it make sense for both Courses and Dates to have the Semester taxonomy?

          Proposed solution

          In addition to linking taxonomies to all other posts with that taxonomy, there is adding the ability to link a taxonomy to a specific custom post as well. This is similar to descriptions for categories, however, taxonomies do not have meta. Posts do.

          This way, when you visit this educational institute’s site and you’re looking at a course, but you’d like to see more about the speaker, you can click one taxonomy link and see all other courses tagged with the speaker, or you can click straight through to the post about that speaker.

          The opposite linking works just as well. You’re looking at a speaker and would like to learn more about a course they teach. The course is already a taxonomy, so you could click and see all the other Speakers who are tagged with this course, i.e. all the Speakers who teach this. Or you could click through to the course itself.

          Obviously this can be done already. Just not automatically, or easily.

          How

          If this were build as a plugin I would create a look-up table linking the taxonomy ID to a post ID. If it were to be incorporated into the core, I would extend `wp_term_taxonomy` with another column that would associate the taxonomy term with the specific custom post ID. A link could be generated with a function like `get_term_post_link()`.

          I think I’ll go ahead and write this plugin now…

          EDIT: It exists!


            How I Optimized My LAMP Server

            I recently switched servers for this site. I moved from Media Temple to Digital Ocean. Think of Digital Ocean as AWS but faster, cheaper, and with great UX. I’ve been meaning to move there for a while, ever since I figured out how to manage my own LAMP stack.

            One benefit of Digital Ocean is their fantastic documentation. So there isn’t much to figure out… But for someone who came from Front-end Development, it’s a bit intimidating to manage your own server. To tell you the truth, I’ve tried this move a few times, but the last time I set up a stack for this site I used SUSE Linux (I don’t know what I was thinking), and the site kept crashing.

            Since then I’ve played with VMWare and got comfortable with setting up my own development server, and moved to CentOS.

            The missing link was optimizing Apache.

            I’m a big fan of This Week in Startups and one of their sponsors is New Relic. If they say something is worth trying, I try it.

            After switching to Digital Ocean I set up New Relic on the new site. Even though I had installed W3 Total Cache on my install, New Relic was still giving me error warnings every 10-15 minutes. Frustrating! True, I AM running a WordPress multisite on the lowest tier, but none of the sites are high traffic. I should be able to do that.

            Well, after digging into New Relics errors I saw that I was using 100% of my my physical memory and 200% of my swap memory. BAD.

            Then I found Jean-Sebastien Morisset’s check_httpd_limits.pl. WOW.

            I updated my httpd.conf with his recommendations and look at the results:

            Physical Memory - New Relic DashboardYou can clearly see when the new settings took effect.

            Here’s the site’s load average:

            Load Average - New Relic Dashboard

            Best part is, since these settings took effect, NO MORE ERROR WARNINGS FROM NEW RELIC!!!

            So, if you read this Jean-Sebastien, thanks for your wonderful tool! And New Relic, thank YOU for your excellent monitoring that pushed me to do this!


              Obamacare Websites “Irresponsibly” Built on WordPress

              Edit: Just wanted to point out. According to the video below, if you go here you can hack ALL OF WordPress! How irresponsible?! Oh yeah, and you can hack Google here.

              I’m a fan of TWIT, I listen to the show weekly–it’s one of my favorite podcasts, in fact. I like it because Leo Laporte is clearly very smart, is knowledgable about tech, and he lands outstanding guests. The clip above, though, is a perfect example of how intelligent people can be wrong.

              “This is the federal one, is also running on WordPress. (laughs)”

              Why wouldn’t it? He really doesn’t explain what issue he has with WordPress, except that you can go to /wp-login.php to get to a login screen. As of writing this post there are over 70 million sites running WordPress. Among them are NBC, TED, TechCrunch, CNN, Time, Dow Jones, and UPS, which are running off WordPress VIP, among many other high-profile sites. WordPress is an elegant platform upon which you can build pretty much anything. In fact, more people are using WordPress as the infrastructure for a web application than they are for purely a blogging engine.

              “Of those who use WordPress, 69% use it only as a CMS (Content Management System); 20% use it as a blog/CMS combo; 6% use it for blogging only; and 7% as an application platform”

              State of the Word 2013, statistics

              So there really is no problem with building your site, even if you are a government health exchange, on top of WordPress. The real problem is who is building that site. I was at a party recently and was shmoozing with a fellow developer who mentioned that his company was forced to use a contractor to build their site. My partner burst out laughing when he said that because of my expressive reaction. Web contractors are notorious for building shoddy sites. I’m not saying every contracted site will be poorly built, but their job is to get the site done and move along, which is not conducive towards quality. Not to mention that a good site is a site that is maintained. Consequently. That is exactly why you should have your site built in WordPress. Whether your site is built in-house, or your site is being contracted, I highly recommend building it off WordPress. WordPress is constantly being developed by a quality open-source community. Open source means that everyone and anyone can dig in and read the code. Sounds a little scary, right? But this actually makes WordPress more secure. I read the WordPress source code for fun in my spare time, I learn a lot that way, and countless other expert developers do the same. When ways to improve are found, they’re included into the next release. If and when security holes are found, patches are released to the community immediately. Can you say that for YOUR site’s infrastructure? If your site is a proprietary site, or maintained by a small team, you can’t say the same. WordPress has been tested by 19% of the internet. If security holes were found regularly, you’d hear about it.

              A good site is a site that is maintained.

              Building off WordPress you can rest assured that your site’s engine will continue to be developed long after your developer has left. YES, your site’s custom theme and plugins will need updating. But that will cost you MUCH less than it would to have a whole new site built. As to the security concerns Leo and his esteemed guests raise. Many developers aren’t aware of all that is needed to properly secure a website,WordPress or not. Especially if your developer is looking over the horizon towards their next gig, being a contractor and all. If you’re concerned about your WordPress site’s security, go ahead and harden your site  right now.


                I May Be Ridiculously Good Looking, But It’s My Choice If I Want To Be Seen Naked

                Walking down the street, no matter how ridiculously hot you are, you can expect that you can keep on the clothing you choose to wear. Even if every passer by wants to see you naked. You may choose to wear long sleeves or a tank top. That is your choice.

                If you go to a house of worship you’ll probably dress more respectfully, if you are in a private institution you may be asked to put on a jacket (you may leave if you don’t want to wear it), and when you go to the beach you might wear a bikini or speedos.

                What you choose to cover up or reveal is a choice you make based on your comfort level, the context of where you are, and your beliefs. But you expect that what you choose to wear, may not be liked, but that choice will be respected.

                If you are in a private home or institution, the owners have a right to ask you to leave if they don’t like how you are dressed. But they don’t have the right to force you to take off your clothing without your consent. That is assault.

                If you would like to go into a public institution there are fears that you may be trying to smuggle contraband in, and you may be searched. In that situation your privacy is being compromised; however, being part of society you are relinquishing that right to an extent to ensure everyone else’s safety. It’s part of the Hobbesian social contract. That is with the assumption that you are giving up only what is necessary and you will be searched with the minimum necessary violation.

                We feel so violated by TSA because our privacy when traveling is being violated wantonly, with unnecessary excess. The same goal could be reached with smarter, better trained, better paid individuals, and less abuse.

                 

                When I joined Facebook it was like a trendy club that all my friends went to. I dressed accordingly. I sought out my friends, and the people I wanted to become my friends. I dressed my sexiest. And acted accordingly.

                Then Facebook announced that the footage from the security cameras in the joint would be auctioned off to the highest bidder.

                Now I still visit Facebook, because everyone I care about is there. But it’s more like going to your third-cousin’s wedding, to which you have no idea why you were invited, and neither do they.

                Sure there are many people there that you know and love dearly. But there is also that distant great aunt who rented you her apartment and upped the rent 40% year over year. Oh yeah, and that lying tattletale colleague is there too. Didn’t you notice? And everything you do or say will be used to SPAM YOU.

                I went to the club I so enjoyed called Facebook, and It turned out I was inappropriately underdressed.

                That was Facebook.

                 

                “What do I have to hide?” they said, when Prism was leaked. “If you have nothing to hide, you have nothing to fear.”

                What if I don’t want to walk down the street naked? I don’t fear it, I AM ridiculously good looking. But I like wearing clothing.

                I like sending an email to a specific person, and know that it is going to them, and not to prying eyes.

                The NSA will claim that their surveillance to falls under the protection of social contract. But that is only true in theory. The fact that some talented high school dropout contractor can look up anyone tells me they did not build the tech with proper checks, regardless of who formally has to sign off.

                There is clearly no consent when everything is placed under a gag order, and everything is collected. That isn’t protecting us, it’s straight out abuse.


                  Add Classes to Menu Items in WordPress

                  Let’s say you want to style each category differently across your site. Go rainbow.

                  How do you target the menu items? If you try targeting them specifically, menu-item-641 might change. What if you’d like to sell your theme? You can’t know in advance that menu-item-641 is going to be “Musings”.

                  So, this is how you do it:

                  Just pop the following code into your functions.php and it will put unique classes on the menus based on the menu names.

                  Enjoy!


                    Keeping WordPress Safe, and Updatable

                    After setting up a WordPress install, there are three magic lines I run from the newly installed WordPress working directory.

                    The first two set the file and folder permissions to the recommended settings for WordPress.

                    The second sets ownership of the files to apache, so that WordPress will be able to update itself.

                    Thought I’d leave these here for anyone who might be digging.

                    IMPORTANT: Do not run these from just any old directory, you can break your server. ONLY run these commands from the directory in which you’ve installed WordPress. Unless you know what you’re doing.