Image Editing with LLMs

Yes, you read that correctly. Large language models (LLMs) can come in handy for not just text and image generation—they can be versatile image editors too. Perhaps you’ve uploaded a photo of yourself to your favorite LLM and asked it to generate a caricature of you or portrayed you as a celebrity being chased by paparazzi. These are LLM-based examples of image editing in which you start with an image, add your text-based prompt to manipulate the image, and the result will hopefully resemble what you had in mind.

I recently came up with another use case for image editing. I was trying to find a high quality image of the Eagles’ Hotel California album cover. I searched the web and found many photos and scans of the album cover, but they all had one major shortcoming—the dark areas in the bottom half of the image had little to no detail. Here are 2 such examples:

I uploaded the images to both Copilot and ChatGPT and entered the following prompt: “These are 2 photos of the cover of the Eagles "Hotel California" album with different exposures and levels of detail in the shadows. Merge them into 1 photo and recover details from the shadows. Significantly boost the shadows so that it is possible to see the trees and bushes. Preserve the fluorescent "Hotel California" words that are superimposed on what looks like a car's side view mirror. Also boost the fluorescent "Hotel California" words so they are more bold. Preserve the original aspect ratio. Reduce overall contrast by making the sky a warmer golden glow and increase overall brightness, especially in the darker shadows.” I got very similar results with both Copilot and ChatGPT, and here is the result from the latter:

As you can see, ChatGPT did a remarkable job of creating the bushes in the lower half of the album cover. I honestly don’t know if it was able to recover detail from the source images or if it generated the bushes from scratch (or perhaps a combination of both). In any case, I was very happy with the recovery of what was otherwise lost detail. It also slightly sharpened the palm trees and did a very nice job of highlighting the “Hotel California” stylized wording while preserving its original look and feel. This was exactly the kind of image quality I had hoped to find in a Hotel California album cover, and although I was unsuccessful with my search, I was thrilled to learn that LLMs could generate the next best thing.

If you’re wondering why I wanted such an image, it’s because I wanted to 3D print the Hotel California album cover, and I wanted to include some detail in the bottom half rather than have it appear pure black. Here is the resultant 3D print:

If you have a 3D printer, you can access the 3D model and print profiles here. I hope this give you ideas for how you can use LLMs to edit photos.

U.S. Passport Online Renewal

My U.S. passport was nearing its 10-year expiration, and I was dreading the renewal process. The last time I renewed, I recall filling out a paper form, jumping through hoops to obtain an “official” passport photo, sending the form and old passport via certified mail, and waiting anxiously for my new passport to arrive while at the same time fretting over the slim possibility that my precious passport could get lost in the mail.

I’m pleased to say that in 2024, the process is much simpler because the U.S. Department of State allows you to renew your passport online if you meet the requirements including:

• You are age 25 or older
• The passport you are renewing is or was valid for 10 years
• The passport you are renewing is either expiring within one year or has expired less than five years ago
• You do not need to expedite your renewal (routine service takes 4-6 weeks)
• You can upload a digital passport photo
• Various other requirements (see link above)

Filling out the online renewal form was easy, and the only information I had to look up were the details related to my current passport (e.g., passport ID and issue date). I was most pleased with the ability to upload a digital passport photo because that was the biggest time-saver in comparison to passport renewal in the old days when we had to provide 2 hardcopy prints that conformed to numerous dimensions as pictured below. Even with my photography skills, the photo requirements were usually too cumbersome for me to bother with, and I’d just pay for passport photo services to make sure they got the dimensions right.

But now you can simply upload your digital photo, and the U.S. Department of State provides a step-by-step overview on its website and accompanies it with a YouTube video. The photo requirements are much simpler now: use a white background, don’t do something stupid (e.g., close your eyes, look away from the camera, open your mouth) and save your original color photo in JPG file format. After uploading your photo, the website will apply conformance checks, automatically crop your photo to a square, and allow you to adjust the cropping if desired. No need to leave your home, print hardcopies, or pay someone to do something that you can easily do with the tiny little supercomputer in your pocket.

Another improvement in the passport renewal process is that you no longer have to mail in your old passport. It is stated, however, that upon submitting your renewal application, your existing passport becomes void and cannot be used for international travel so make sure you plan ahead. After paying the processing fee (which was $130 for a passport book at the time of this writing) and submitting my passport renewal application, I received an email confirmation and was also given the option to sign up for additional notifications about the status of my application. Easy peasy lemon squeezy!

On Professional Photography

I’ve always considered myself a photography hobbyist or enthusiast. I enjoy the process of taking photos, processing the RAW files, and sometimes playing around with HDR, panoramic stitching, masking, and/or converting my photos into home movies. Sometimes people compliment me on my photos by saying, “You’re a professional photographer!” While I appreciate such comments and recognize that they usually mean that my photos may appear to be of “professional” quality, I usually reply by stating that a professional photographer actually makes money from their photos, unlike me.

Every now and then, I post what I consider to be one of my better photos to a site called 500px. While Facebook, Instagram, and other social media sties feature plenty of photos, 500px tends to be used by enthusiasts and “professional” photographers, so the composition and image quality of the photos you’ll see there are significantly better. One feature of 500px is that users can sell their photos. Today I received an email notification that I made a sale!

While $1.73 is not a whole lot of money, I now satisfy a very loose definition of what a professional photographer is: I made money from my photography! And technically speaking, if you look back at my sales history, I became a professional photographer back in June 2014:

However, my first sale was a little unusual. I sent my parents a link to my gallery to share some of my favorite photos, and my mom actually bought one of them. I told her that I could have sent her a copy of the photo for free, but that wasn’t the point—she just wanted to show support. After all, that’s what moms do.

Today’s purchase by Getty Images, however, is completely legitimate. Clearly I need to use a stricter definition of “professional photographer”—it makes more sense to define it as someone who depends on earnings from their photography, in whole or at least in part, to make a living. By that definition, I am certainly not a professional photographer, but I still enjoy being a hobbyist.

My Photo and Video Stats

I have a large external hard drive onto which I am able to fit my entire collection of digital photos and videos. In browsing some of the administrator functions associated with the external hard drive, I came across this page:



I noticed that as of today, I have a grand total of 110,118 pictures and 3,909 videos. I am assuming that "pictures" is inclusive of .JPG as well as raw (e.g., Canon's .CR2 format) and photoshop (.PSD) file. Soon I will be adding .HEIC files to my collection, as iOS 11 has replaced .JPG with .HEIC as its new default image file format--I assume that these files will fall under the "pictures" tally as well.

Finally, I assume that "videos" is inclusive of .MOV (the native format in Apple devices), .MTS (the native AVCHD format in my Canon camcorder), and .MP4 (the common format that is almost universally compatible with modern devices like desktops, laptops, mobile phones, tablets, and TV set top boxes). If so, then those numbers should be representative of my complete collection. A final note... while iOS 11 will be replacing the H.264 codec with HEVC, I believe videos will continue to have a .MOV container, so the file extension should not change.

So those are my stats. Do you know how many photos and videos you have in your collection?

Bizarre Love Triangle - Photos for iOS, Google Photos, and Photos for macOS

Has this ever happened to you?

In the last 2 years since we both bought a 64 GB iPhone 6 Plus, my wife has rarely deleted photos and videos from her phone. Today she received the above error message. Given that everyone has a finite amount of space on their phones, I thought I'd share my "phonetography" workflow on iOS and macOS. This of course applies to both iPhones and iPads.

1) Take pictures, lots of them. Why not? Storage space is cheap, and memories are precious. Fire away. Turn on HDR and save both the regular and HDR versions. Record video in the maximum resolution and maximum frame rate. View them in Photos for iOS.

2) Periodically sync to Google Photos. As long as you have a Google account, you can upload an unlimited number of photos to your account, provided that you allow Google to apply some compression to your photos. While you currently get 15 GB if you save the original full resolution images from your phones, I think a small amount of compression (no noticeable difference in image quality in my opinion) is worth having unlimited storage. Two reasons for this: most people take rather mediocre smartphone photos and this option is just a backup for hard drive disasters.

3) After syncing to Google Photos, sync photos to Photos for macOS (formerly iPhoto). This will include your original unaltered photos, plus any in-phone edits you might have applied. Also backup these photos to at least 1 (preferably more) external hard drives to insulate against hard drive failure on your main Mac.

4) Only after completing steps 2 and 3, go back and delete photos that you do not need immediately accessible on your phone. Keep the ones you absolutely want to have on your phone, knowing that you have full-resolution backups on Photos for macOS (and external hard drives) as well as your entire library available in the cloud on Google Photos which can be accessed anywhere that you have an internet connection. If you are judicious about preserving a selected number of "keepers" on your phone, you should rarely run into trouble with running out of space on your phone--at least not due to photos.

If you have a workflow and archival programs you'd like to share, leave a comment!

Facebook 'Mirror Boy' Controversy

Last night I posted a photo of myself on Facebook.  The subject of the photo was me when I was approximately 1 year old, standing naked in front of a mirror.  After all, it was Thursday, and given that it is one of my favorite childhood photos, I thought the photo was worthy of a #TBT hashtag.  This morning I received the following email from Facebook:

Upon visiting Facebook via desktop web browser, it forced me to login despite having previously stored my username and password in my browser (the same applied to my Facebook app for iOS).  Following successful login, rather than displaying my typical feed of UFC events, nature videos, and stories illustrating that Donald Trump possesses the emotional intelligence of a chick pea, I was presented with this message:

So take that, Facebook--that's me in my birthday suit for the whole world to see.  There was certainly no pornographic intent, and clearly I had consented to posting the image since I had carefully scanned and edited (rotating, cropping, tone adjustment, and other minor edits only--nothing that would require surgical intervention to correct) the photo before posting it to my own personal Facebook feed which I make visible only to my personal contacts.  If you read the Facebook Community Standards (referenced in the above image) language on nudity, the wording all falls under the section of "Encouraging respectful behavior" where nudity, hate speech, and violence and graphic content policies are described.  The 'Mirror Boy' photo was also not intended to be disrespectful, nor do I think that most people would perceive it to be disrespectful.  I wonder if Blogger (owned by Google) would censor this image too?  I guess I'll find out after I post this blog article.

On a related topic, a controversy recently unfolded about Facebook censoring the 'Napalm Girl' photo, and following much public criticism of Facebook's policy, the photo was reinstated across the site.  You can find articles about it here, here, and here, to highlight a few of the many stories written about it.  Coincidentally, both the 'Napalm Girl' and 'Mirror Boy' photos were taken in 1972.  Should I petition to reinstate the 'Mirror Boy' photo on my Facebook feed?

Annual Photo and Video Storage

Out of curiosity, I charted my year-over-year hard drive storage of photos and videos. This was a relatively simple exercise since I store all of my photos and videos in parent folders named by year, so all I had to do was have the operating system calculate each folder's size. I did make some minor adjustments--for example, in 2013 I made a video of a vacation that I took back in 2002, and I stored the video in the 2002 folder, but since I created the content in 2013, I re-allocated that storage space from 2002 to 2013. So essentially the following graphic depicts storage space based on year of content creation:

As you can see, there is an upward trend of storage space consumption. This is not surprising because with each new camera purchased, along came more megapixels. Additionally, in late 2009 when my daughter was born, I began shooting in RAW (actually RAW + JPG). I believe that the combination of baby photos and the addition of RAW files sufficiently explains the jump in storage space between 2009 and 2010.

Not depicted in the figure are some milestones related to video capture. In 10/2009, I purchased a Polaroid DVG-1080P camcorder to capture videos of my then soon-to-be-born daughter. In 5/2011, I upgraded my camcorder to a Canon VIXIA HF M301 which performed much better in low light and had a far superior image stabilization feature.

There appears to be slightly higher storage space consumed in 2005 as compared with the next few years, and I attribute that to our wedding and honeymoon in April and May of 2005 where we not surprisingly generated lots of photo and video content.

Prior to 2005, I shot digital photos with a Canon PowerShot A70 (7/2003 to 6/2005), my first digital camera. However, the storage space was negligible on the above scale. I also used a Canon EOS Rebel S to shoot 35 mm film from approximately mid-1992 to mid-2003, and while I did scan all my negatives and save them as JPG files, that storage space was also negligible.

For the first half of 2016, I am at around 250 GB of storage, so extrapolation to a full year would land me at around 500 GB.  Therefore, it looks like I am leveling off at approximately half a terabyte annually. I suppose this is useful for me to know so I can estimate when I will need to buy additional external hard drives to back up my photo and video content.

Lee Family 2015 Year In Review

 

Here is our Lee Family 2015 Year In Review. It is our 10th edition, and all prior editions are available here.  Also visit Victor's Picasa Web Albums site and YouTube channel.

Editing with Apple Photos 1.0

I decided to sign up for Apple's OS X Beta Program to download the Mac OS X Yosemite 10.10.3 beta so I could get my hands on the new Photos app.  Photos has received significant attention in the Apple community because it is the much-anticipated successor to both iPhoto and Aperture, Apple's consumer and professional photography workflow applications respectively.  Here are my first impressions.

After updating from Mac OS X 10.10.2 to 10.10.3, the Photos app was already installed and sitting in my Applications folder.  I also have both iPhoto and Aperture installed, and those were untouched by the installation, although the iPhoto icon in my dock was replaced by Photos.  A Feedback Assistant application was also installed and placed in my dock, and the name is self-explanatory.

Upon first launch of Photos (version 1.0, build 205.44.0), I was provided the options to start with a blank Photos library or to import a library from iPhoto or Aperture.  I imported my iPhoto library, and even though I only had approximately 2,200 photos (which is probably less than 1% of all the content in my entire library), the import process took about 15 minutes, and I did not have any other programs running in the background.  Once you begin using Photos to store new photos and edit them, you are essentially committing to a new source of truth, as your photo libraries between iPhoto/Aperture and Photos will begin to diverge.  In fact, upon launching iPhoto after my initial Photos import, it warned me that any changes that I made would not be reflected in my Photos library which I thought would be helpful for users who might not have understood that.

I was primarily interested in the photo editing options, so I will review them here.  The navigation options are very simple and clean, true to my perception of Apple's philosophy about user interfaces.  Upon selecting a photo, an "Edit" button appears, and upon clicking it you are presented with the following options:

• Enhance.  This appears to attempt enhance your photo, and there are no options--you either do it or you don't.  I suspect that depending on your photo, you may or may not get good results, but I suppose it would be worth it for people to try it out.
• Rotate.  By default, clicking this option will rotate your image counterclockwise, but you can right-click to reveal options to rotate clockwise.
• Crop.  Clicking this option reveals an intuitive interface that allows for both cropping and rotation.  An Aspect option allows you to select freeform or any number of common predefined aspect ratios which is nice.  You can also flip your photo along its vertical axis.  Curiously, there is an Auto option which seems to do all of this for you, but I haven't played around with it enough to judge whether it is useful or not.  I would not plan to use it very often.
• Filters. This option simply presents a variety of 8 preset filters such as Mono, Noir, Fade, and Instant to produce a variety of effects.  Personally I am not a big fan of filters so do not plan to use them.
• Adjust. Clicking this option reveals 3 groups of adjustment sliders.  Each group displays an AUTO option when you mouse over the group, but there are also finer controls which I describe below.
• Light. Under this group, there are sliders for Exposure, Highlights, Shadows, Brightness, Contrast, and Black Point.  While each one can be adjusted individually by sliding or typing in a value, adjusting the Light group slider results in simultaneous changes to all 6 parameters in that group.  My initial impression is that this may seem very useful for quick edits, and I'd have to play around with this a bit more to see if the Light group slider produced the desired effect or if I would still find myself needing to adjust the individual sliders in that group.
• Color. Under this group, we have sliders for Saturation, Contrast, and Cast.  Similar to above, you can drag individual sliders or the Color group slider.
• Black & White. Under this group, we have sliders for Intensity, Neutrals, Tone, and Grain.  While you can drag the Black & White group slider, it has no effect on the sliders within the group.
• Retouch. This option provides what seems to be a hybrid of spot removal and clone stamping features, for those of you who are familiar with Adobe Photoshop terminology.  There is a Size slider that allows you to define the diameter of the area you wish to edit, and there are simply instructions to "Click & drag over spots to remove" and "Option-click to choose the source area".  I prefer Photoshop's options to let me either remove spots or to clone stamp because they produce slightly different results, although I can understand why Apple would cater to the masses by simplifying these two similar functions into one.

I was hoping that there would be options for lens corrections (e.g. to remove barrel distortion) and noise reduction (very helpful for photos shot in low light) options since I frequently use them when processing photos in Photoshop or Lightroom.  There are also many other features of Photos such as RAW processing, iCloud Photo Sharing, etc. that I have not reviewed, as this post was focused on the editing capabilities of Photos.  For a quick and general overview, see David Pogue's review of Apple Photos.

In conclusion, Photos feels very much like a replacement for iPhoto.  Its options are elegant and simple which will probably cater to the needs of most novice photographers.  However, I doubt that many Aperture users will migrate to Photos--instead, they will move their photo assets to Lightroom or other similar applications.  For me personally, I have historically only used iPhoto to store photos that I want to sync with my iPhone and iPad (whereas I've used Photoshop and Lightroom for RAW conversions and photo editing) and will probably do the same with Photos upon its official release.