What Effect Does Shape Have in a Art Work

• Journal List
• HHS Author Manuscripts
• PMC6749614

Proc ACM Int Conf Multimed. Author manuscript; available in PMC 2019 Sep xviii.

Published in final edited form every bit:

PMCID: PMC6749614

NIHMSID: NIHMS1041151

On Shape and the Computability of Emotions

Abstract

We investigated how shape features in natural images influence emotions aroused in human beings. Shapes and their characteristics such as roundness, angularity, simplicity, and complication have been postulated to affect the emotional responses of homo beings in the field of visual arts and psychology. However, no prior enquiry has modeled the dimensionality of emotions aroused by roundness and angularity. Our contributions include an in-depth statistical analysis to empathize the relationship betwixt shapes and emotions. Through experimental results on the International Affective Picture Organization (IAPS) dataset we provide testify for the significance of roundness-angularity and simplicity-complication on predicting emotional content in images. We combine our shape features with other country-of-the-art features to evidence a gain in prediction and classification accuracy. We model emotions from a dimensional perspective in order to predict valence and arousal ratings which have advantages over modeling the traditional discrete emotional categories. Finally, we distinguish images with strong emotional content from emotionally neutral images with high accuracy.

Keywords: Human being Emotion, Psychology, Shape Features, Algorithms, Experimentation, Man Factors, H.3.one [Information Storage and Retrieval], Content analysis and indexing, I.4.7 [Image Processing and Figurer Vision], Feature measurement

1. INTRODUCTION

The study of human visual preferences and the emotions imparted past diverse works of art and natural images has long been an agile topic of research in the field of visual arts and psychology. A computational perspective to this problem has interested many researchers and resulted in manufactures on modeling the emotional and artful content in images [10, eleven, 13]. Withal, at that place is a wide gap between what humans can perceive and feel and what can be explained using electric current computational image features. Bridging this gap is considered the "holy grail" of computer vision and the multimedia community. There accept been many psychological theories suggesting a link between man affective responses and the low-level features in images apart from the semantic content. In this work, we try to extend our understanding of some of the low-level features which have non been explored in the study of visual affect through extensive statistical analyses.

In contrast to prior studies on image aesthetics, which intended to judge the level of visual appeal [10], we try to leverage some of the psychological studies on characteristics of shapes and their result on human emotions. These studies indicate that roundness and complication of shapes are fundamental to understanding emotions.

• Roundness - Studies [four, 21] indicate that geometric properties of visual displays convey emotions like anger and happiness. Bar et al. [five] confirm the hypothesis that curved contours lead to positive feelings and that sharp transitions in contours trigger a negative bias.

• Complexity of shapes - As enumerated in diverse works of art, humans visually prefer simplicity. Whatever stimulus pattern is always perceived in the most simplistic structural setting. Though the perception of simplicity is partially subjective to individual experiences, it can also be highly affected by ii objective factors, parsimony and orderliness. Parsimony refers to the minimalistic structures that are used in a given representation, whereas orderliness refers to the simplest way of organizing these structures [3].

These findings provide an intuitive understanding of the low-level image features that motivate the affective response, but the small calibration of studies from which the inferences have been drawn makes the results less convincing. In order to brand a fair comparison of observations, psychologists created the standard International Affective Picture System (IAPS) [xv] dataset by obtaining user ratings on three basic dimensions of touch on, namely valence, arousal, and authorisation (Effigy ane). Still, the computational work on the IAPS dataset to understand the visual factors that affect emotions has been preliminary. Researchers [9, 11, 18, 23, 25, 26] investigated factors such as color, texture, composition, and simple semantics to sympathise emotions, but have non quantitatively addressed the outcome of perceptual shapes. The study that did explore shapes by Zhang et al. [27] predicted emotions evoked by viewing abstract art images through depression-level features like color, shape, and texture. However, this work simply handles abstract images, and focused on the representation of textures with fiddling accountability of shape.

Example images from IAPS (The International Affective Picture show Organisation) dataset [15]. Images with positive affect from left to right, and high arousal from bottom to top.

The current work is an attempt to systematically investigate how perceptual shapes contribute to emotions angry from images through modeling the visual properties of roundness, angularity and simplicity using shapes. Dissimilar edges or boundaries, shapes are influenced by the context and the surrounding shapes influence the perception of any individual shape [3]. To model these shapes in the images, the proposed framework statistically analyzes the line segments and curves extracted from stiff continuous contours. Investigating the quantitative relationship betwixt perceptual shapes and emotions angry from images is non-trivial. Commencement, emotions aroused by images are subjective. Thus, individuals may not have the same response to a given prototype, making the representation of shapes in complex images highly challenging. 2d, images are not composed of elementary and regular shapes, making information technology difficult to model the complexity existing in natural images [three].

Leveraging the proposed shape features, the current work attempts to automatically distinguish the images with stiff emotional content from emotionally neutral images. In psychology, emotionally neutral images refer to images which evoke very weak or no emotions in humans.

Also, the electric current study models emotions from a not- categorical or discrete emotional perspective. In previous work, emotions were distinctly classified into categories like acrimony, fear, cloy, amusement, awe, and contentment, amongst others. This paper is, to our knowledge, the first to predict emotions aroused from images past adopting a dimensional representation (Figure 2). Valence represents the positive or negative aspect of human being emotions, where common emotions, like joy and happiness, are positive, whereas anger and fear are negative. Arousal describes the human physiological state of being reactive to stimuli. A higher value of arousal indicates higher excitation. Potency represents the controlling nature of the emotion. For instance, anger tin be more controlling than fear. Researchers [ii, 12, 28] take investigated the emotional content of videos through the dimensional approach. Their accent was on the accommodation of the change in features over time rather than low-level feature improvement. However, static images, with less data, are oft more than challenging to interpret. Depression-level features demand to be punctuated.

Dimensional representation of emotions and the location of chiselled emotions in these dimensions (Valance, Arousal, and Dominance).

This work adopts the dimensional approaches of emotion motivated by contempo studies in psychology, which argued for the strengths of dimensional approaches. According to Bradley and Lang [half-dozen], categorized emotions do not provide a i-to-ane relationship between the content and emotion of an image since participants perceive different emotions in the aforementioned epitome. This highlights the utility of a dimensional arroyo, which controls for the intercorrelated nature of human emotions aroused by images. From the perspective of neuroscience studies, it has been demonstrated that the dimensional arroyo is more than consistent with how the brain is organized to procedure emotions at their most basic level [14, 17]. Dimensional approaches also let the separation of images with strong emotional content from images with weak emotional content.

In summary, our main contributions are:

• Nosotros systematically investigate the correlation between visual shapes and emotions angry from images.

• We quantitatively model the concepts of roundness-angularity and simplicity-complexity from the perspective of shapes using a dimensional approach.

• We distinguish images with stiff emotional content from those with weak emotional content.

The rest of the newspaper is organized as follows, Section two provides a summary of previous piece of work. Section 3 introduces some definitions and themes which recur throughout the paper. The overall framework followed by details of the perceptual shape descriptors are described in Section 4. Experimental results and in-depth analyses are presented in Section 5. We conclude in Department vi.

two. RELATED Piece of work

Previous work [eleven, 26, 18] predicted emotions aroused past images mainly through preparation classifiers on visual features to distinguish chiselled emotions, such equally happiness, anger, and distressing. Depression-level stimuli such as colour and composition take been widely used in computational modeling of emotions. Affective concepts were modeled using colour palettes, which showed that the bag of colors and Fisher vectors (i.e., higher club statistics about the distribution of local descriptors) were effective [9]. Zhang et al. [27] characterized shape through Zernike features, edge statistics features, object statistics, and Gabor filters. Emotion-histogram and bag-of-emotion features were used to classify emotions past Solli et al. [24]. These emotion metrics were extracted based on the findings from psycho-physiological experiments indicating that emotions can exist represented through homogeneous emotion regions and transitions among them.

The commencement piece of work that comprehensively modeled chiselled emotions, Machajdik and Hanbury [18] used color, texture, composition, content, and semantic level features such as number of faces to model eight discrete emotional categories. Besides the eight basic emotions, to model categorized emotions, adjectives or discussion pairs were used to stand for human emotions. The earliest work based on the Kansei system employs 23 word pairs (due east.1000., like-dislike, warmcool, cheerful-gloomy) to plant the emotional space [23]. Along the same lines, researchers enumerated more than word pairs to achieve a universal, distinctive, and comprehensive representation of emotions in Wang et al. [25]. Yet, the aforementioned approaches of emotion representation ignore the interrelationship amidst types of emotions.

three. CONCEPT Interpretation

This work captures emotions evoked by images past leveraging shape descriptors. Shapes in images are hard to capture, mainly due to the perceptual and merging boundaries of objects which are often not like shooting fish in a barrel to differentiate using even country-of-the-art sectionalization or contour extraction algorithms. In gimmicky estimator vision literature [7, xx], there are a number of statistical representations of shape through characteristics similar the straightness, sinuosity, linearity, circularity, elongation, orientation, symmetry, and the mass of a bend. We chose roundness-angularity and simplicity-complication characteristics considering they take been found previously by psychologists to influence the affect of human beings through controlled human subject studies. Symmetry is too known to consequence emotion and aesthetics of images [22]. However, quantifying symmetry in natural images is challenging.

To go far more convenient to introduce the shape features proposed, this section defines the 4 terms used: line segments, angles, continuous lines, and curves. The framework for extracting perceptual shapes through lines and curves is derived from [eight]. The contours are extracted using the algorithm in [1], which used color, texture, and brightness of each image for contour extraction. The extracted contours are of different intensities and signal the algorithm's conviction on the presence of edges. Considering the temporal resolution of our vision organization, we adopted a threshold of 40%. Example results are presented in Figures iii, ​ iv, ​ 5, and ​ 6. Pixels with an intensity college than 40% are treated equally, which results in the binary contour map presented in the 2d cavalcade. The terminal three columns evidence the line segments, continuous lines, and curves.

Perceptual shapes of images with high valance.

Perceptual shapes of images with low valance.

Perceptual shapes of images with high arousal.

Perceptual shapes of images with depression arousal.

Line segments -

Line segments refer to brusk straight lines generated by fitting nearby pixels. We generated line segments from each prototype to capture its construction. From the construction of the epitome, we propose to interpret the simplicity-complexity. We extracted locally optimized line segments by connecting neighboring pixels from the contours extracted from the epitome [xvi].

Angles -

Angles in the image are obtained by calculating angles between each of any two intersecting line segments extracted previously. According to Julian Hochberg'southward theory [three], the number of angles and the number of dissimilar angles in an image tin can be effectively used to describe its simiplicity-complexity. The distribution of angles also indicates the degree of angularity of the image. A high number of acute angles makes an image more angular.

Continuous lines -

Continuous lines are generated by connecting intersecting line segments having the same orientations with a small margin of fault. Line segments of inconsistent orientations can be categorized as either corner points or points of inflexion. Corner points, shown in Figure 7(a), refer to angles that are lower than 90 degrees. Inflexion points, shown in Figure 7(b), refer to the midpoint of two angles with contrary orientations. Continuous lines and the degree of curving can be used to interpret the complexity of the image.

The corner bespeak and signal of inflexion.

Curves -

Curves are a subset of continuous lines, the collection of which are employed to measure the roundness of an image. To achieve this, we consider each bend as a department of an ellipse, thus we use ellipses to fit continuous lines. Fitted curves are represented by parameters of its corresponding ellipses.

4. CAPTURING EMOTION FROM SHAPE

For decades, numerous theories take been promoted that are focused on the relationship between emotions and the visual characteristics of simplicity, complication, roundness, and angularity. Despite these theories, researchers accept notwithstanding to resolve how to model these relationships quantitatively. In this department, we propose to utilise shape features to capture those visual characteristics. Past identifying the link betwixt shape features and emotions, we are able to determine the human relationship between the aforementioned visual characteristics and emotions.

We now present the details of the proposed shape features: line segments, angles, continuous lines, and curves. A total of 219 shape features are summarized in Table ane.

Table 1:

Summary of shape features.

Category	Short Name	#
Line Segments	Orientation	60
	Length	11
	Mass of the image	4
Continuous Lines	Degree of curving	xiv
	Length span	9
	Line count	4
	Mass of continuous lines	4
Angles	Angle count	3
	Angular metrics	35
Curves	Fitness	14
	Circularity	17
	Area	8
	Orientation	xiv
	Mass of curves	4
	Pinnacle round curves	18

4.1. Line segments

Psychologists and artists have claimed that the simplicity-complexity of an image is determined not only by lines or curves, but also by its overall structure and back up [3]. Based on this idea, we employed line segments extracted from images to capture their structure. Especially, nosotros used the orientation, length, and mass of line segments to determine the complexity of the images.

Orientation -

To capture an overall orientation, we employed statistical measures of minimum (min), maximum (max), 0.75 quantile, 0.25 quantile, the difference betwixt 0.75 quantile and 0.25 quantile, the difference between max and min, sum, total number, median, hateful, and standard deviation (we volition subsequently refer to these as {statistical measures}), and entropy. We experimented with both 6- and xviii-bin histograms. The unique orientations were measured based on the two histograms to capture the simplicity- complication of the epitome.

Among all line segments, horizontal lines and vertical lines are known [3] to be static and to represent the feelings of calm and stability within the image. Horizontal lines propose peace and at-home, whereas vertical lines point forcefulness. To capture the emotions evoked by these characteristics, we counted the number of horizontal lines and vertical lines through an 18-bin histogram. The orientation θ, of horizontal lines autumn within 0° < θ < 10° or 170° < θ < 180°, and 80° < θ < 100° for vertical lines.

Length -

The length of line segments reflects the simplicity of images. Images with simple construction might use long lines to fit contours, whereas complex contours have shorter lines. We characterized the length distribution past calculating the {statistical measures} of lengths of line segments within the image.

Mass of the image -

The centroid of line segments may indicate associated relationships amongst line segments within the visual blueprint [3]. Hence, nosotros calculate the mean and standard deviation of the 10 and y coordinates of the line segments to find the mass of each image.

Some of the example images and their features are presented in Figures 8 and ​ 9. Effigy 8 presents the ten lowest mean values of the length of line segments. The commencement row shows the original images, the second row shows the line segments extracted from these images and the third row shows the 18-bin histogram for line segments in the images. The 18 bins refer to the number of line segments with an orientation of [−90 + ten(i — i), −90 + 10i) degrees where i ∈ {one, 2,…, 18}. Similarly, Figure 9 presents the x highest mean values of the length of line segments.

Images with low mean value of the length of line segments and their associated orientation histograms. The commencement row is the original images; the 2nd row shows the line segments; and the third row shows the xviii-bin histogram for line segments in the images.

Images with loftier hateful value of the length of line segments and their associated orientation histograms. The get-go row is the original images; the second row shows the line segments; and the tertiary row shows the 18-bin histogram for line segments in the images.

These two figures indicate that the length or the orientation cannot be examined separately to decide the simplicity-complexity of the image. Lower hateful values of the length of line segments might refer to either simple images such equally the get-go four images in Figure 8 or highly circuitous images such as the concluding 4 images in that effigy. The histogram of the orientation of line segments helps united states of america to distinguish the complex images from simple images by examining variation of values in each bin.

4.2. Angles

Angles are of import elements in analyzing the simplicity-complexity and the angularity of an epitome. We capture the visual characteristics from angles through ii perspectives.

• Angle count - We starting time calculate the two quantitative features claimed past Julian Hochberg, who has attempted to define simplicity (he used the value-laden term "figural goodness") via information theory: "The smaller the amount of information needed to define a given organization equally compared to the other alternatives, the more likely that the figure will be so perceived" [3]. Hence this minimal information structure is captured using the number of angles and the percentage of unique angles in the prototype.

• Angular metrics - We use the {statistical measures} to extract athwart metrics. We also calculate the 6- and 18-bin histograms on angles and their entropies.

Some of the example images and features are presented in Figures 10 and ​ xi. Images with lowest and highest number of angles are shown along with their corresponding contours in Figure 10. These examples show promising relationships between angular features and simplicity-complexity of the paradigm. Case results for the histogram of angles in the epitome are presented in Effigy 11. The eighteen bins refer to the number of line segments with an orientation in [10(i— 1), 10i) degrees where i ε {1, 2,…, 18}.

Images with highest and lowest number of angles.

The distribution of angles in images.

iv.3. Continuous lines

We try to capture the degree of curvature from continuous lines, which has implications for the simplicity-complexity of images. We also calculated the number of continuous lines, which is the third quantitative feature specified by Julian Hochberg [three]. For continuous lines, open/closeness are factors affecting the simplicity-complexity of an image. In the following, we focus on the adding of the degree of curving, the length span value, and the number of open lines and closed lines. The length span refers to the highest Euclidean distance amid all pairs of points on the continuous lines.

Length Span ( l ) = max p i ∈ l , p j ∈ 50 EuclideanDist ( p i , p j ) ,

(one)

where [p1,p ₂ , …,p _{Due north} } are the points on continuous line l.

• Caste of curving - We calculated the degree of curving of each line every bit

  Caste of Curving ( l ) = Length Span ( l ) / Northward ,

  (2)

  where N is the number of points on continuous line 50.

  To capture the statistical characteristics of contiguous lines in the image, we calculated the {statistical measures}. We as well generated a 5-bin histogram on the degree of curving of all continuous lines (Figures 12 and ​ 13).

  

  Images with highest degree of curving.

  

  Images with lowest caste of curving.

• Length span - We used {statistical measures} for the length span of all continuous lines.

• Line count - Nosotros counted the total number of continuous lines, the total number of open up lines, and the total number of closed lines in the image.

4.4. Curves

Nosotros used the nature of curves to model the roundness of images. For each curve, we calculated the extent of fit to an ellipse as well as the parameters of the ellipse such as its expanse, circularity, and mass of curves. The curve features are explained in detail below.

• Fettle, expanse, circularity - The fettle of an ellipse refers to the overlap between the proposed ellipse and the curves in the image. The area of the fitted ellipse is also calculated. The circularity is represented by the ratio of the pocket-size and major axes of the ellipses. The angular orientation of the ellipse is likewise measured. For each of the measures, nosotros used the {statistical measures} and entropies of the histograms as the features to depict the roundness of the image.

• Mass of curves - We used the mean value and standard deviation of (x, y) coordinates to describe the mass of curves.

• Summit round curves - To brand full apply of the discovered curves and to depict roundness, we included the fitness, surface area, circularity, and mass of curves for each of the top iii curves.

To examine the relationship between curves and positivenegative images, we calculated the average number of curves in terms of values of circularity and fitness on positive images (i.eastward., the value is higher than half-dozen in the dimension of valance) and negative images (i.due east., the value is lower than 4.v in the dimension of valance).

The results are shown in Tables two and ​ 3. Positive images have more curves with 60% – 100% fettle to ellipses and higher boilerplate curve count.

Table 2:

Average number of curves in terms of the value of fitness in positive and negative images.

	(0.eight, one]	(0.vi, 0.8]	(0.4, 0.half-dozen]	(0.2, 0.four]
Positive imgs	2.12	nine.33	5.7	2.68
Negative imgs	1.42	seven.v	five.02	2.73

Tabular array 3:

Average number of curves in terms of the value of circularity in positive and negative images.

	(0.8,ane]	(0.6, 0.8]	(0.four, 0.6]	(0.2, 0.four]
Positive imgs	0.96	2.56	5.1	11.2
Negative imgs	0.73	2.19	4	9.75

v. EXPERIMENTS

To demonstrate the human relationship between proposed shape features and the felt emotions, the shape features were utilized in three tasks. First, we distinguished images with strong emotional content from emotionally neutral images. 2nd, nosotros fit valence and arousal dimensions using regression methods. We then performed classification on discrete emotional categories. The proposed features were compared with the features discussed in Machajdik et al. [18], and overall accuracy was quantified by combining those features. Forrad option and Primary Component Analysis (PCA) strategies were employed for characteristic selection and to discover the best combination of features.

five.i. Dataset

Nosotros used ii subsets of the IAPS [xv] dataset, which were developed by examining homo melancholia responses to color photographs with varying degrees of emotional content. The IAPS dataset contains 1,182 images, wherein each image is associated with an empirically derived hateful and standard deviation of valance, arousal, and potency ratings.

Subset A of the IAPS dataset includes many images with faces and human bodies. Facial expressions and body linguistic communication strongly affect emotions aroused past images, slight changes of which might atomic number 82 to an opposite emotion. The proposed shape features are sensitive to faces hence we removed all images with faces and human bodies from the scope of this study. In experiments, we only considered the remaining 484 images, which we labeled as Subset A. To provide a better agreement of the ratings of the dataset, we analyzed the distribution of ratings within valence and arousal, as shown in Effigy fourteen. We also calculated average variations of ratings in each rating unit (i.eastward., 1–ii, 2–iii, … , 7–8). Valence ratings between 3 and 4, and 6 and vii, have the maximum variance for single images. Similarly, arousal ratings between 4 and 5 varied the most.

Distribution of ratings in IAPS.

Subset B are images with category labels (with detached emotions), generated by Mikels [nineteen]. Subset B includes eight categories namely, anger, disgust, fright, sadness, amusement, awe, contentment, and excitement, with 394 images in total. Subset B is a ordinarily used dataset, hence we used it to benchmark our nomenclature accuracy with the results mentioned in Machajdik et al. [18].

5.ii. Identifying Stiff Emotional Content

Images with strong emotional content have very loftier or very depression valance and arousal ratings. Images with values around the hateful values of valance and arousal lack emotions and wered used as samples for emotionally neutral images.

Based on dimensions of valance and arousal respectively, we generated two sample sets from Subset A. In Set 1, images with valence values college than 6 or lower than 3. 5 were considered images with strong emotional content and the residual to represent emotionally neutral images. This resulted in 247 emotional images and 237 neutral images. Similarly, images with arousal values college than 5.five or lower than 3. vii were divers as emotional images, and others as neutral images. With similar thresholds, we obtained 239 emotional images and 245 neutral images in Set ii.

We used the traditional Support Vector Machines (SVM) with radial footing role (RBF) kernel to perform the classification task. We trained SVM models using the proposed shape features, Machajdik's features, and combined (Machajdik's and shape) features. Training and testing were performed by dividing the dataset uniformly into training and testing sets. As we removed all images with faces and human bodies, nosotros did not consider facial and pare features discussed in [18]. We used both forwards selection and PCA methods to perform feature selection. In the frontwards selection method, nosotros used the greedy strategy and accumulated one feature at a time to obtain the subset of features that maximized the classification accuracy. The seed features were also chosen at random over multiple iterations to obtain improve results. Our analyses showed that the frontwards selection strategy achieved greater accuracy for Set 2, whereas PCA performed better for Set up 1 (Figure 15). The feature comparison showed that the combined (Machajdik's and shape) features achieved the highest nomenclature accuracy, whereas individually the shape features lonely were much stronger than the features from [xviii] (Machajdik's features). This outcome is intuitive since emotions evoked by images cannot be well represented by shapes lone and can definitely be bolstered by other prototype features including their color composition and texture. By analyzing valence and arousal ratings of the correctly classified images, nosotros observed that very complex/simple, circular and angular images had strong emotional content and loftier valence values. Simple structured images with very low degrees of curving also tends to portray strong emotional content as well equally to accept high arousal values.

Classification accuracy (%) for emotional images and neutral images (Set 1 and Gear up 2 are defined in Department five.2).

By analyzing the individual features for classification accuracy we found that line count, fettle, length span, degree of curving, and the number of horizontal lines achieved the best nomenclature accuracy in Set 1. Fitness and line orientation were more than ascendant in Set 2.

We present a few case images, which were wrongly classified based on the proposed shape features in Figures 16 and ​ 17. The misclassification can be explained as a shortcoming of the shape features in agreement the semantics. Some of the images generated extreme emotions based on epitome content irrespective of the low-level features. As well the semantics, our performance was also limited past the functioning of the contour extraction algorithm.

Examples of misclassification in Set 1. The iv rows are original images, image contours, line segments, and continuous lines.

Examples of misclassification in Gear up two. The four rows are original images, images contours, line segments, and continuous lines.

5.3. Fitting the Dimensionality of Emotion

Emotions can be represented by word pairs, as previously done in [23]. However, some emotions are difficult to label. Modeling basic emotional dimensions helps in alleviating this problem. Nosotros represented emotion as a tuple consisting of valence and arousal values. The values of valence and arousal were in the range of (ane, 9). In order to predict the values of valence and arousal we proposed to learn a regression model for either dimension separately.

We used SVM regression with RBF kernel to model the valance and arousal values using shape, Machajdik'south features, as well equally the combination of features. The mean squared error (MSE) was computed for each of the private features likewise as combined for both valence and arousal values separately. The MSE values are shown in Figure 18(a). These figures prove that the valance values were modeled more accurately past Machajdik's features than our shape features. Arousal was well modeled by shape features with a mean squared error of 0.nine. Notwithstanding, the combined characteristic functioning did non show any improvements. The results indicated that visual shapes provide a stronger cue in understanding the valence as opposed to the combination of color, texture, and composition in images.

Experimental results. (a) Mean squared mistake for the dimensions of valance and arousal. (b) Accuracy for the classification chore.

We also computed the correlation between quantified private shape features and valence-arousal ratings. The college the correlation, the more than relevant the features were. Through this process we institute that angular count, fettle, circularity, and orientation of line segments showed higher correlations with valance, whereas angle count, angle metrics, straightness, length span, and orientation of curves had higher correlations with arousal.

5.4. Classifying Categorized Emotions

To evaluate the relationship between shape features and emotions on discrete emotions, nosotros classified images into one of the eight categories, anger, disgust, fearfulness, sadness, entertainment, awe, delectation, and excitement. We followed Machajdik et al. [18] and performed 1-vs-all classification to compare and benchmark our classification accuracy. The nomenclature results are reported in Figure 18(b). We used SVM to assign the images to one of the viii classes. The highest accuracy was obtained by combining Machajdik'south with shape features. We also observed a considerable increase in the classification accuracy by using the shape features lonely, which proves that shape features indeed capture emotions in images more effectively.

In this experiment, nosotros also built classifiers for each of the shape features. Each of the shape features listed in Tabular array 4 accomplished a nomenclature accuracy of xxx% or higher.

Table 4:

Pregnant features to emotions.

Emotion	Features
Angry	Circularity
Disgust	Length of line segments
Fear	Orientation of line segments and angle count
Sadness	Fitness, mass of curves, circularity, and orientation of line segments
Entertainment	Mass of curves and orientation of line segments
Awe	Orientation of line segments
Excitement	Orientation of line segments
Delectation	Mass of lines, angle count, and orientation of line segments

6. CONCLUSIONS

We investigated the computability of emotion through shape modeling. To achieve this goal, we first extracted contours from complex images, and and then represented contours using lines and curves extracted from images. Statistical analyses were conducted on locally meaningful lines and curves to correspond the concept of roundness, angularity, and simplicity, which take been postulated as playing a primal role in evoked emotion for years. Leveraging the computational representation of these concrete stimulus backdrop, nosotros evaluated the proposed shape features through three tasks: distinguishing emotional images from neutral images; classifying images co-ordinate to categorized emotions; and plumbing fixtures the dimensionality of emotion based on proposed shape features. We have achieved an improvement over the state-of-the-art solution [18]. Nosotros likewise attacked the problem of modeling the presence or absenteeism of stiff emotional content in images, which has long been overlooked. Separating images with strong emotional content from emotionally neutral ones can assist in many applications including improving the functioning of keyword based image retrieval systems. We empirically verified that our proposed shape features indeed captured emotions in the images. The area of understanding emotions in images is still in its infancy and modeling emotions using depression-level features is the first step toward solving this problem. We believe our contribution takes us closer to agreement emotions in images. In the future, we hope to expand our experimental dataset and provide stronger prove of established relationships between shape features and emotions.

Acknowledgments

J. Li and J. Z. Wang are as well affiliated with the National Science Foundation. This textile is based upon work supported by the Foundation. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the Foundation.

7. REFERENCES

[one] Arbelaez P, Maire M, Fowlkes C, and Malik J. Contour detection and hierarchical image segmentation. IEEE Trans. on Pattern Analysis and Auto Intelligence, 33(5):898–916, 2011. [PubMed] [Google Scholar]

[2] Arifin S and Cheung PYK. A computation method for video segmentation utilizing the pleasure-arousal-dominance emotional data. In ACM MM, pages 68–77, 2007. [Google Scholar]

[3] Arnheim R. Fine art and visual perception: A psychology of the artistic eye. 1974. [Google Scholar]

[4] Aronoff J. How we recognize angry and happy emotion in people, places, and things. Cross-Cultural Research, 40(1):83–105, 2006. [Google Scholar]

[5] Bar 1000 and Neta M. Humans prefer curved visual objects. Psychological Science, 17(8):645–648, 2006. [PubMed] [Google Scholar]

[vi] Bradley MM and Lang PJ. The international affective picture organization(IAPS) in the study of emotion and attending. In Handbook of Emotion Elicitation and Assessment, pages 29–46, 2007. [Google Scholar]

[7] Brandt S, Laaksonen J, and Oja E. Statistical shape features in content-based prototype retrieval. In ICPR, pages 1062–1065, 2000. [Google Scholar]

[eight] Chia A, Rajan D, Leung M, and Rahardja S. Object recognition by discriminative combinations of line segments, ellipses and appearance features. IEEE Trans. on Pattern Analysis and Motorcar Intelligence, 34(9):1758–1772, 2011. [PubMed] [Google Scholar]

[nine] Csurka Thou, Skaff S, Marchesotti 50, and Saunders C. Building look & feel concept models from color combinations. The Visual Computer, 27(12):1039–1053, 2011. [Google Scholar]

[ten] Datta R, Joshi D, Li J, and Wang JZ. Studying aesthetics in photographic images using a computational approach. In ECCV, pages 288–301, 2006. [Google Scholar]

[xi] Datta R, Li J, and Wang JZ. Algorithmic inferencing of aesthetics and emotion in natural epitome: An exposition. In ICIP, pages 105–108, 2008. [Google Scholar]

[12] Hanjalic A and Xu LQ. Affective video content representation and modeling. IEEE Trans. on Multimedia, vii(1):143–154, 2005. [Google Scholar]

[13] Joshi D, Datta R, Fedorovskaya Eastward, Luong QT, Wang JZ, Li J, and Luo J. Aesthetics and emotions in images. IEEE Signal Processing Magazine, 28(v):94–115, 2011. [Google Scholar]

[xiv] Lang PJ, Bradley MM, and Cuthbert BN. Emotion, motivation, and anxiety: Brain mechanisms and psychophysiology. Biological Psychiatry, 44(12):1248–1263, 1998. [PubMed] [Google Scholar]

[xv] Lang PJ, Bradley MM, and Cuthbert BN. International affective picture organisation: Affective ratings of pictures and instruction manual In Technical Report A-8, Academy of Florida, Gainesville, FL, 2008. [Google Scholar]

[16] Leung MK and Yang Y-H. Dynamic two-strip algorithm in bend fitting. Blueprint Recognition, 23(one– 2):69–79, 1990. [Google Scholar]

[17] Lindquist KA, Wager TD, Kober H, Bliss-Moreau E, and Barrett LF. The encephalon ground of emotion: A meta-analytic review. Behavioral and Brain Sciences, 173(iv):1–86, 2011. [Google Scholar]

[xviii] Machajdik J and Hanbury A. Melancholia paradigm classification using features inspired by psychology and art theory. In ACM MM, pages 83–92, 2010. [Google Scholar]

[19] Mikel J, Fredrickson BL, Larkin GR, Lindberg CM, Maglio SJ, and Reuter-Lorenz PA. Emotional category data on images from the international affective picture system. Behavior Enquiry Methods, 37(4):626–630, 2005. [PMC free article] [PubMed] [Google Scholar]

[xx] Mingqiang Y, Kidiyo K, and Joseph R. A survey of shape characteristic extraction techniques. Pattern Recognition, pages 43–xc, 2008. [Google Scholar]

[21] Reber R, Schwarz North, and Winkielman P. Processing fluency and artful pleasure: Is beauty in the perceiver's processing experience? Personality and, Social Psychology Review, 8(iv):364–382, 2004. [PubMed] [Google Scholar]

[22] Schiffman Hour. Sense and Perception: An Integrated Approach. 1990. [Google Scholar]

[23] Shibata T and Kato T. Kansei image retrieval system for street landscape-bigotry and graphical parameters based on correlation of two epitome systems. In International Conference on Systems, Homo, and Cybernetics, pages 274–252, 2006. [Google Scholar]

[24] Solli Chiliad and Lenz R. Color based numberless-of-emotions. LNCS, 5702:573–580, 2009. [Google Scholar]

[25] Wang HL and Cheong LF. Affective understanding in motion picture. IEEE Trans. on Circuits and Systems for Video Engineering, sixteen(6):689–704, 2006. [Google Scholar]

[26] Yanulevskaya V, Van Gemert JC, Roth G, Herbold AK, Sebe N, and Geusebroek JM. Emotional valence categorization using holistic image features. In ICIP, pages 101–104, 2008. [Google Scholar]

[27] Zhang H, Augilius E, Honkela T, Laaksonen J, Gamper H, and Alene H. Analyzing emotional semantics of abstruse fine art using low-level epitome features. In Advances in Intelligent Data Analysis, pages 413–423, 2011. [Google Scholar]

[28] Zhang SL, Tian Q, Huang QM, Gao W, and Li SP. Utilizing affective analysis for efficient pic browsing. In ICIP, pages 1853–1856, 2009. [Google Scholar]

fewingsthavatabot.blogspot.com

Source: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6749614/

Share :