A number of new capabilities in iOS 15 use artificial intelligence to assess the content of photos taken with an iPhone.
Even while object recognition technology has been available for a while, Apple claims that its implementation differs because it takes place on the device itself rather than on a remote cloud server. On top of that, it’s a part of the OS.
For example, users can take a photo of a page from a book and then copy and paste the content from that page into a document or a note, which is one of the most useful applications of this new technology.
Another benefit is that phone numbers and email addresses written down, like on a business card, can be recognised by the app and called or sent more easily by phone or email
Handwritten text can also be turned into notes with this feature, as demonstrated by Apple in June when it announced the addition of the capability to its iOS operating system.
Here’s how it works:
- Take a photo with text you want to copy into a document. For this example, we’re using a recipe from a cookbook.
- Go into the Photos app and select the photo with text you want to copy. This feature also works from the camera roll, inside the camera app.
- Press down on the text you want to copy with your finger, the same way you would in an Apple Note or text message. Two text selector dots should appear on the photo.
- Stretch the text selection cursors to cover all the text you want to copy. Press “copy.”
- Go into the word processor of your choice, tap on the text field, and paste.
- The new feature also works directly in a word processor like the Notes app.
- Tap on the screen and pick the icon in the pop-up that looks like three lines of text in a square box. That’s the Live Text icon.
- It will bring a camera window up on the bottom half of the screen. Point it at text and the software will automatically recognize it.
- Apple’s software will highlight the text it wants to insert into the app inside the photo window.
- Press the blue “Insert” button when the software has selected the right chunk of text.
- You can also scan text directly from the iPhone’s Camera app.
- Point the camera at words and if the software recognizes text, it will display the three-lines-of-text icon in the bottom right of the viewfinder, or top-right in landscape.
- Press the button. It will turn yellow and bring up a window dedicated to the text in the photo.
- You can select this text and copy-paste it into any document.